NPS-AM-1 2-204 


ACQUISITION  RESEARCH 
SPONSORED  REPORT  SERIES 


Naval  Ship  Maintenance:  An  Analysis  of  the  Dutch 
Shipbuilding  Industry  Using  the  Knowledge  Value  Added, 
Systems  Dynamics,  and  Integrated  Risk  Management 

Methodologies 

10  October  2012 
by 

Dr.  David  N.  Ford,  Research  Associate  Professor 

Graduate  School  of  Business  &  Public  Policy 

Dr.  Thomas  J.  Housel,  Professor,  and 
Dr.  Johnathan  Mun,  Research  Professor 

Graduate  School  of  Operational  &  Information  Sciences 

Naval  Postgraduate  School 


Approved  for  public  release,  distribution  is  unlimited. 
Prepared  for:  Naval  Postgraduate  School,  Monterey,  California  93943 


ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY 

NAVAL  POSTGRADUATE  SCHOOL 


Report  Documentation  Page 


Form  Approved 
OMB  No.  0704-0188 


Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 
VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  OMB  control  number. 


1.  REPORT  DATE 

10  OCT  2012 


2.  REPORT  TYPE 


3.  DATES  COVERED 

00-00-2012  to  00-00-2012 


5a.  CONTRACT  NUMBER 


5b.  GRANT  NUMBER 


5c.  PROGRAM  ELEMENT  NUMBER 


5d.  PROJECT  NUMBER 


5e.  TASK  NUMBER 


5f.  WORK  UNIT  NUMBER 


4.  TITLE  AND  SUBTITLE 

Naval  Ship  Maintenance:  An  Analysis  of  the  Dutch  Shipbuilding  Industry 
Using  the  Knowledge  Value  Added,  Systems  Dynamics,  and  Integrated 
Risk  Management  Methodologies 

6.  AUTHOR(S) 


7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES)  8.  PERFORMING  ORGANIZATION 

Naval  Postgraduate  School, Graduate  School  of  Business  &  Public  report  number 

Policy ,555  Dyer  Road,  Ingersoll  Hall, Monterey, CA, 93943 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES)  10.  SPONSOR/MONITOR’S  ACRONYM(S) 

11.  SPONSOR/MONITOR’S  REPORT 
NUMBER(S) 

12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release;  distribution  unlimited 

13.  SUPPLEMENTARY  NOTES 

14.  ABSTRACT 

Initiatives  to  reduce  the  cost  of  ship  maintenance  have  not  yet  realized  the  normal  cost-reduction  learning 
curve  improvements.  One  explanation  is  the  lack  of  recommended  technologies.  Damen,  a  Dutch 
shipbuilding  and  service  firm,  has  incorporated  similar  technologies  and  is  developing  others  to  improve  its 
operations.  The  research  team  collected  data  on  Dutch  ship  maintenance  operations  and  used  them  to  build 
three  types  of  computer  simulation  models  of  ship  maintenance  and  technology  adoption.  The  results  were 
analyzed  and  compared  with  previously  developed  modeling  results  of  U.S.  Navy  ship  maintenance  and 
technology  adoption.  Adopting  3D  PDF  alone  improves  ROI  significantly  more  than  adopting  a  logistics 
package  alone  and  adding  both  technologies  improves  ROI  more  than  adding  either  technology  alone. 
Adoption  of  the  technologies  would  provide  cost  benefits  far  in  excess  of  not  using  the  technologies  and 
there  were  marginal  benefits  in  sequentially  implementing  the  technologies  over  immediately  implementing 
them.  There  are  a  number  of  issues  in  comparing  the  results  with  previous  research  but  the  potential 
benefits  of  using  the  technologies  are  very  high  in  both  cases.  Implications  for  acquisition  practice  include 
the  need  for  careful  analysis  and  selection  from  among  a  variety  of  available  information  technologies  and 
the  recommendation  for  a  phased  development  and  implementation  approach  to  manage  uncertainty. 


15.  SUBJECT  TERMS 


16.  SECURITY  CLASSIFICATION  OF: 


a.  REPORT 

unclassified 


b.  ABSTRACT 

unclassified 


c.  THIS  PAGE 

unclassified 


17.  LIMITATION  OF 

18.  NUMBER 

ABSTRACT 

OF  PAGES 

Same  as 

109 

Report  (SAR) 

19a.  NAME  OF 
RESPONSIBLE  PERSON 


Standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANSI  Std  Z39-18 


The  research  presented  in  this  report  was  supported  by  the  Acquisition  Research 
Program  of  the  Graduate  School  of  Business  &  Public  Policy  at  the  Naval 
Postgraduate  School. 

To  request  defense  acquisition  research,  to  become  a  research  sponsor,  or  to  print 
additional  copies  of  reports,  please  contact  any  of  the  staff  listed  on  the  Acquisition 
Research  Program  website  (www.acquisitionresearch.net). 


ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY 

NAVAL  POSTGRADUATE  SCHOOL 


Abstract 


Initiatives  to  reduce  the  cost  of  ship  maintenance  have  not  yet  realized  the 
normal  cost-reduction  learning  curve  improvements.  One  explanation  is  the  lack  of 
recommended  technologies.  Damen,  a  Dutch  shipbuilding  and  service  firm,  has 
incorporated  similar  technologies  and  is  developing  others  to  improve  its  operations. 
The  research  team  collected  data  on  Dutch  ship  maintenance  operations  and  used 
them  to  build  three  types  of  computer  simulation  models  of  ship  maintenance  and 
technology  adoption.  The  results  were  analyzed  and  compared  with  previously 
developed  modeling  results  of  U.S.  Navy  ship  maintenance  and  technology 
adoption.  Adopting  3D  PDF  alone  improves  ROI  significantly  more  than  adopting  a 
logistics  package  alone  and  adding  both  technologies  improves  ROI  more  than 
adding  either  technology  alone.  Adoption  of  the  technologies  would  provide  cost 
benefits  far  in  excess  of  not  using  the  technologies  and  there  were  marginal  benefits 
in  sequentially  implementing  the  technologies  over  immediately  implementing  them. 
There  are  a  number  of  issues  in  comparing  the  results  with  previous  research  but 
the  potential  benefits  of  using  the  technologies  are  very  high  in  both  cases. 
Implications  for  acquisition  practice  include  the  need  for  careful  analysis  and 
selection  from  among  a  variety  of  available  information  technologies  and  the 
recommendation  for  a  phased  development  and  implementation  approach  to 
manage  uncertainty. 

Keywords:  Technology  adoption,  ship  maintenance,  Laser  scanning 
technology,  Collaborative  product  lifecycle  management 
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Executive  Summary 


There  have  been  a  series  of  initiatives  designed  to  reduce  the  cost  of  ship 
maintenance.  SHIPMAIN,  and  its  derivatives,  was  one  of  the  initiatives  designed  to 
improve  ship  maintenance  performance  within  the  Navy  by  standardizing  processes 
in  order  to  take  advantage  of  learning-curve  cost  savings.  However,  these  process 
improvement  initiatives  have  not  yet  realized  the  normal  cost-reduction  learning 
curve  improvements  for  common  maintenance  items  for  a  series  of  common 
platform  ships.  One  explanation  is  that  the  initial  instantiation  of  SHIPMAIN  did  not 
include  two  recommended  technologies,  3-dimensional  laser  scanning  technology 
(3D  LST)  and  collaborative  product  lifecycle  management  (CPLM),  that  were 
deemed  necessary  by  the  creator  of  SHIPMAIN  for  ensuring  the  success  of  the  new 
standardized  approach  (i.e.,  normal  learning  curve  cost  savings). 

Damen,  a  large  Dutch  shipbuilding  and  service  firm  has  incorporated  similar 
technologies  and  is  developing  others  to  improve  its  operations.  In  addition,  the 
Royal  Dutch  Navy  performs  all  of  its  own  ship  maintenance  in  a  single  yard  and 
operation,  and,  we  extrapolated  and  compared  the  potential  benefits  of  similar 
technologies  with  similar  projections  for  the  U.S.  Navy  ship  maintenance  processes. 
These  organizations  provide  a  source  of  relatively  reliable  data  on  operations  that 
are  comparable  to  those  performed  by  the  U.S.  Navy.  One  variation  on  the  3D  LST 
technology  was  the  use  of  3D  PDF  by  the  Damen  company.  The  3D  PDF  technology 
was  roughly  comparable  to  the  potential  use  of  3D  LST  and  provided  a  baseline 
approximation  for  the  use  of  3D  LST  in  projecting  the  future  value  of  the  technology. 

This  study  compared  the  Dutch  experience  with  projections  for  the  use  of  the 
two  technologies  in  the  U.S.  Navy.  Because  the  technologies  have  not  been  fully 
implemented  in  the  Dutch  shipbuilding  and  ship  maintenance  venue,  we 
extrapolated  from  the  partial  implementation  of  CPLM  and  3D  PDF  in  order  to 
compare  potential  projections  of  possible  improvements  in  both  the  Dutch  and  U.S. 
cases. 
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The  current  work  addresses  the  following  questions: 


How  are  the  Dutch  using  and  preparing  to  adopt  advanced  technologies,  such  as  3D 
LST  and  CPLM,  in  shipbuilding  and  maintenance? 

What  are  the  potential  changes  in  ROIs  provided  by  the  adoption  of  these  advanced 
technologies? 

How  do  those  potential  returns  compare  with  projected  estimates  of  returns  on 
technology  adoption  of  3D  LST  and  CPLM  in  the  U.S.  Navy? 

The  research  team  collected  data  on  Dutch  ship  maintenance  operations  and 
used  them  to  build  three  types  of  computer  simulation  models  of  ship  maintenance 
and  technology  adoption.  The  approach  included  use  of  the  knowledge  value  added 
(KVA)  models  of  return  on  technology  investments  in  those  operations,  system 
dynamics  models  (based  on  the  KVA  preliminary  ROI  results)  of  ship  maintenance 
operations,  and  integrated  risk  management  (IRM)  models  of  implementation  plans 
for  the  technology  adoption.  The  results  were  then  analyzed  and  compared  with 
previously  developed  modeling  results  of  U.S.  Navy  ship  maintenance  and 
technology  adoption. 

The  linear  ROI  projections  from  adopting  various  iterations  of  the  partial 
CPLM  tool  (i.e. ,  only  the  logistics  package)  and  the  3D  PDF  tool  in  the  Dutch  context 
demonstrated  the  advantages  of  adopting  both  technologies  over  either  technology 
alone,  compared  to  a  baseline  without  either  technology.  Adopting  3D  PDF  alone 
improves  ROI  significantly  more  than  adopting  a  logistics  package  alone  (100% 
improvement  >  46%  improvement)  and  adding  both  technologies  improves  ROI 
more  than  adding  either  technology  alone  (239%  improvement  >  42%  improvement 
or  100%  improvement),  suggesting  that  there  may  be  synergy  between  the 
technologies.  This  is  also  supported  by  the  139%  improvement  gained  by  adding 
logistics  if  3D  PDF  is  already  in  place.  These  results  were  then  used  to  forecast  the 
benefits  of  various  adoption  options  for  the  tools  using  the  IRM  methodology. 

The  results  of  the  IRM  analysis  provided  forecasts  of  the  benefits  of  various 
options  for  implementing  the  technologies  separately  or  in  combination.  The  results 
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(shown  in  the  following  figure)  indicated  that  adoption  of  the  technologies  would 
provide  cost  benefits  far  in  excess  of  not  using  the  technologies.  In  addition,  the 
results  indicated  that  there  were  marginal  benefits  in  sequentially  implementing  the 
technologies  over  immediately  implementing  them.  Given  the  long  cycle  for 
organizations  to  benefit  from  technology  adoption,  it  might  be  better  to  adopt  the 
technologies  immediately. 


ANALYSIS  RESULTS 

Strategy  A  As-ls 

Startegy  B  30  PDF  &  LOGI  STIC  S  TC  (111  PLEM  ENT  NOW) 
Strategy  C  30  PDF  1 N  TC  0 N  L  Y  (IM  PLEM  ENT  NOW) 

Startegy  D  LOGI  STIC  S  MQDLI LE  ON  LY  (I  MPLE  M  E  NT  NOW) 
Strategy  E  30  PDF  AMD  LOGI  ST1CS  TC  (PHASED  SEQUENTIAL) 
Startegy  F  30  PDF  IN  TC  ONLY  (PHASED  SEQUENTIAL) 
Strategy  G  LOGISTICS  MODULE  ONLY  (PHASED  SEQUENTIAL) 


Strategic  Real  Options 

KVA  ROI  KVA  ROK  Real  Options  ROI  Volatility 


35.00% 

135.00% 

$31,903,557 

35.00% 

82.67% 

273.32% 

373.82% 

$154,163,306 

273.53% 

87.71% 

135.06% 

235.06% 

$96,330,730 

137.25% 

54.82% 

77.28% 

177.28% 

$81,009,562 

91.66% 

80.24% 

273.32% 

373.82% 

$156,569,744 

282.88% 

87.71% 

135.06% 

235.06% 

$97,416,808 

138.79% 

54.82% 

77.28% 

177.28% 

$84,456,260 

95.56% 

80.24% 

Net  Differential:  Strategy  E  over  Strategy  B 
Net  Differential:  Strategy  E  over  Strategy  F 


S2.405.938 

$59,152,936 


Comparing  these  results  with  U.S.  Navy  results  offers  some  partially  confirming 
evidence  for  the  prior  research  that  projected  the  benefits  of  adopting  CPLM  and  3D 
LST  technologies  for  ship  maintenance.  There  are  a  number  of  issues  in  making 
these  comparisons  that  must  be  noted  given  the  differences  in  the  size  of  the  two 
countries’  ship  maintenance  operations  and  in  the  extent  of  implementation  of  the 
two  types  of  technologies.  However,  the  comparisons  have  validity  when  these 
issues  are  accounted  for,  and  the  potential  benefits  of  using  the  technologies  are 
very  high  in  both  cases. 


The  scenarios  have  some  similarities.  All  overall  returns  on  investment  are 
positive  and  large.  This  supports  the  adoption  of  advanced  technologies  such  as  3D 
LST,  3D  PDF  models,  and  CPLM  to  improve  the  efficiency  of  resource  use.  The 
scenarios  also  have  potentially  significant  differences.  The  preparation  for 
maintenance  phases  of  the  U.S.  scenario  has  a  much  larger  technology-adopted 
return  on  investment  than  the  maintenance  implementation  phases  of  the  U.S. 
scenario  or  the  Dutch  scenario  (2,019%  »  201%  or  274%).  Several  factors  could 
explain  this  difference.  A  real  options  approach  was  applied  to  investigate  the 
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impacts  of  technology  selection  and  implementation  strategies  on  project  value.  The 
adoption  of  3D  PDF  models  and  a  logistics  management  package  within  a  CPLM 
environment  was  found  to  provide  the  greatest  value. 

Implications  for  acquisition  practice  include  the  need  for  careful  analysis  and 
selection  from  among  a  variety  of  available  information  technologies  and  the 
recommendation  for  a  phased  development  and  implementation  approach  to 
manage  uncertainty. 
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I.  Introduction 


The  current  cost-constrained  environment  within  the  federal  government  and 
DoD  requires  a  defensible  approach  to  cost  reductions  without  compromising  the 
capability  of  core  defense  processes  and  platforms.  Due  to  this  environment,  defense 
leaders  today  must  maintain  and  modernize  the  U.S.  armed  forces  to  retain 
technological  superiority  while  simultaneously  balancing  defense  budget  cost 
constraints  and  extensive  military  operational  commitments.  At  the  same  time,  defense 
leaders  must  navigate  a  complex  information  technology  (IT)  acquisition  process. 
Maintenance  programs  play  a  critical  role  in  meeting  these  DoD  objectives.  One  such 
core  process  that  is  central  to  U.S.  naval  operations  is  the  ship  maintenance  process. 
This  process  alone  accounts  for  billions  of  dollars  in  the  U.S.  Navy’s  annual  budget. 
There  have  been  a  series  of  initiatives  designed  to  reduce  the  cost  of  this  core  process, 
including  ship  maintenance.  SHIPMAIN,  and  its  derivatives,  was  one  of  the  initiatives 
designed  to  improve  ship  maintenance  performance  within  the  Navy  by  standardizing 
processes  in  order  to  take  advantage  of  learning  curve  cost  savings.  Figure  1  provides 
a  notional  picture  of  this  phenomenon. 


Figure  1.  Ship  Maintenance  Learning  Curve 
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However,  these  process  improvement  initiatives  have  not  yet  realized  the  normal 
cost-reduction  learning  curve  improvements  for  common  maintenance  items  for  a  series 
of  common  platform  ships.  One  explanation  is  that  the  initial  instantiation  of  SHIPMAIN 
did  not  include  two  recommended  technologies,  3-dimensional  laser  scanning 
technology  (3D  LST)  and  collaborative  product  lifecycle  management  (CPLM),  that  were 
deemed  necessary  by  the  creator  of  SHIPMAIN  for  ensuring  the  success  of  the  new 
standardized  approach  (i.e.,  normal  learning  curve  cost  savings).  Previous  research 
(Ford,  Housel,  &  Mun,  2011)  indicates  that  adding  these  technologies  may  help 
SHIPMAIN,  or  its  derivatives,  to  capture  the  potential  saving.  But  the  technologies  have 
not  been  implemented  to  date  in  the  ship  maintenance  processes. 

However,  Damen,  a  large  shipbuilding  and  service  firm  has  incorporated  similar 
technologies  and  is  developing  others  to  improve  its  operations.  In  addition,  the  Royal 
Dutch  Navy  performs  all  of  its  own  ship  maintenance  in  a  single  yard  and  operation.  In 
the  current  study  the  potential  benefits  of  similar  technologies  are  extrapolated  and 
compared  with  similar  projections  for  U.S.  Navy  ship  maintenance  processes.  These 
organizations  provide  a  source  of  relatively  reliable  data  on  operations  that  are 
comparable  to  those  performed  by  the  U.S.  Navy. 

A.  Problem  Description 

Previous  research  on  the  potential  use  of  3D  LST  and  CPLM  technology  in  U.S. 
Navy  ship  maintenance  (e.g.,  Komoroski,  2005;  Ford,  Housel,  &  Mun,  2011)  estimated 
the  impacts  on  processes  due  to  technology  adoption.  Changes  such  as  reengineering 
ship  maintenance  processes,  the  sizes  of  reductions  in  cycle  times,  and  workforce 
requirements  are  examples  of  model  portions  that  required  modelers  to  make 
assumptions  about  the  potential  impacts  of  these  technologies  in  modeling  projected 
results.  While  the  previous  work  has  provided  defensible  estimates  of  potential 
improvements  (in  returns  on  investment,  ROI)  and  cost  savings,  the  validity  and 
usefulness  of  these  models  has  been  limited  by  the  lack  of  comparative  data  on  ship 
maintenance  processes  and  technology  investments,  and  of  their  potential  impacts  on 
performance.  Therefore,  the  acquisition  of  data  on  Dutch  naval  fleet  maintenance 
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processes  and  comparison  of  those  data  with  previous  U.S.  Navy  results  was  a  critical 
next  step  in  improving  U.S.  naval  technology  acquisition  decision-making,  in  particular 
with  regard  to  ship  maintenance. 

To  be  valuable,  the  data  source  or  sources  for  this  work  had  to  have  several 
critical  similarities  with  U.S.  naval  ship  maintenance  processes.  The  data  source  had  to 
consider  technological  innovation  and  the  adoption  of  advanced  technologies  to  be  an 
important  part  of  its  naval  maintenance  acquisition  strategy.  The  data  source  or  sources 
had  to  be  large  enough  to  support  continuous  ship  maintenance  operations  because  the 
intermittent  stopping  and  restarting  of  operations  would  not  be  consistent  with  important 
assumptions  of  the  modeling  approach.  Finally,  the  data  source  had  to  be  accessible, 
willing  to  share  the  data,  and  willing  to  allow  us  to  obtain  the  new  data  required  for  our 
modeling  approach.  These  and  other  criteria  limited  the  potential  pool  of  sources  to 
nations  or  large  industrial  ship  maintenance  organizations  that  were  on  good  terms  with 
the  United  States,  advanced  enough  in  their  operations  to  compare  with  those  of  the 
U.S.  Navy,  progressive  enough  in  their  strategies  to  include  continuous  technology 
adoption,  and  willing  to  share  data  and  information  that  is  often  considered  essential  for 
national  security  or  competitive  advantage.  Damen  Industries  and  the  Royal  Dutch  Navy 
(RDN)  met  most  of  these  criteria  and  were  willing  to  meet  our  requirements  for  data 
acquisition  and  sharing. 

The  current  work  addresses  the  following  questions: 

•  How  are  the  Dutch  using  and  preparing  to  adopt  advanced 
technologies,  such  as  3D  LST  and  CPLM,  in  shipbuilding  and 
maintenance? 

•  What  are  the  potential  changes  in  ROIs  provided  by  the  adoption  of 
these  advanced  technologies? 

•  How  do  those  potential  returns  compare  with  projected  estimates  of 
returns  on  technology  adoption  of  3D  LST  and  CPLM  in  the  U.S. 

Navy? 
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B.  Research  Methodology  and  Background 


The  traditional  ROI  equation  is  typically  expressed  as  (Revenue  -  Investment)  / 
Investment,  which  represents  the  productivity  ratio  of  output  (i.e.,  Revenue  in  ROI  + 
Input  or  Investment  Cost  in  ROI).  Accomplishing  this  analysis  in  a  nonprofit  environment 
presents  challenges  because  there  is  no  actual  revenue  generated.  Cost  savings  from 
reductions  in  manpower  requirements  (i.e.,  time  allocated  to  employee  workload  for 
various  tasks)  is  available  to  provide  the  impact  on  the  denominator  of  the  ship 
maintenance  efforts.  However,  the  knowledge  value  added  (KVA)  methodology  also 
allows  for  generation  of  a  quantifiable  surrogate  for  revenue  in  the  form  of  common 
units  of  output  described  in  terms  of  units  of  learning  time.1  Specifically,  the  KVA 
methodology  allowed  the  study  team  to  quantify  the  knowledge  embedded  in  the  new 
processes  to  use  in  generating  common  units  of  output  estimates. 

The  KVA  analysis  provided  the  basic  ROI  estimates  critical  in  forecasting  the 
future  value  of  various  automation  options  within  an  optimized  portfolio  over  time  using 
the  integrated  risk  management  (IRM)  framework  and  supporting  toolset. 

The  research  team  collected  data  on  Dutch  ship  operations  as  described  in  the 
Data  Collection  section  and  used  it  to  build  three  types  of  computer  simulation  models 
of  ship  maintenance  and  technology  adoption:  knowledge  value  added  (KVA)  models  of 
return  on  technology  investments  in  those  operations,  system  dynamics  models  (based 
on  the  KVA  preliminary  ROI  results)  of  ship  maintenance  operations,  and  integrated  risk 
management  models  of  implementation  plans  for  technology  adoption.  The  results  were 
then  analyzed  and  compared  with  previously  developed  modeling  results  of  U.S.  Navy 
ship  maintenance  and  technology  adoption.  In  what  follows,  we  review  the  three 
approaches  to  projecting  the  potential  cost  benefits  of  adopting  the  technologies, 
beginning  with  a  general  review  of  the  KVA,  SD,  and  IRM  approaches.  This  is  followed 
by  the  projected  results  from  applying  these  approaches  to  assess  the  impacts  of  the 


1  KVA  can  provide  other  means  for  describing  outputs  in  common  units,  such  as  lines  of  code  (controlling 
for  complexity  per  line  of  code)  and  process  instructions  (controlling  for  complexity  per  instruction),  as  well 
as  other  means. 
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two  technologies.  A  comparison  of  the  Dutch  and  U.S.  naval  maintenance  results  is 
provided  followed  by  the  results  of  the  IRM  forecasts. 

Knowledge  value  added  (KVA)  measures  the  value  provided  by  human  capital 
and  IT  assets  by  an  organization,  process,  or  function  at  the  subprocess  level  (Figure 
2).  It  monetizes  the  outputs  of  all  assets,  including  intangible  knowledge  assets. 
Capturing  the  value  embedded  in  an  organization’s  core  processes,  employees,  and  IT 
enables  the  actual  cost  and  revenue  of  a  product  or  service  to  be  calculated. 


Figure  2.  Measuring  Output 
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Figure  3.  Comparison  of  Traditional  Accounting  Versus 

Process-Based  Costing 


Total  value  is  captured  in  two  key  metrics:  return  on  investment  (ROI)  and  return 
on  knowledge  (ROK).  While  ROI  is  the  traditional  financial  ratio,  ROK  identifies  how  a 
specific  process  converts  existing  knowledge  into  producing  outputs  so  decision-makers 
can  quantify  costs  and  measure  value  derived  from  investments  in  human  capital 
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assets.  A  higher  ROK  signifies  better  utilization  of  knowledge  assets.  If  IT  investments 
do  not  improve  the  ROK  value  of  a  given  process,  steps  must  be  taken  to  improve  that 
process’s  function  and  performance. 

Table  1.  KVA  Metrics 


Metric 

Description 

Type 

Calculation 

Return  on  Knowledge 
(ROK) 

Basic  productivity,  cash¬ 
flow  ratio 

Subcorporate,  process-level 
performance  ratio 

(Outouts-Benefits  in  Common  Units) 

Cost  to  Produce  Output 

Return  on  Investment 
(ROI) 

Same  as  ROI  at  the  sub¬ 
corporate,  process  level 

Traditional  investment 
finance  ratio 

(Revenue-Investment  Cost) 

Investment  cost 

The  goal  is  to  determine  which  core  processes  provide  the  highest  ROIs  and 
ROKs,  and  to  make  suggested  process  improvements  based  on  the  results.  In  the 
current  work,  KVA  is  used  to  measure  the  benefits  of  technology  adoption  in  Dutch  ship 
maintenance.  This  analysis  provides  a  means  to  check  the  reliability  of  prior  studies’ 
estimates  of  the  potential  ROI  core  process  improvements  from  using  CPLM,  3D  and 
LST  in  ship  maintenance  core  processes  in  the  U.S.  Navy  yards. 

C.  System  Dynamics 

The  system  dynamics  methodology  applies  a  control  theory  perspective  to  the 
design  and  management  of  complex  human  systems.  System  dynamics  combines 
servo-mechanism  thinking  with  computer  simulation  to  create  insights  about  the 
development  and  operation  of  these  systems.  It  is  one  of  several  established  and 
successful  approaches  to  systems  analysis  and  design  (Flood  &  Jackson,  1991;  Lane  & 
Jackson,  1995;  Jackson,  2003).  Forrester  (1961)  develops  the  methodology’s 
philosophy,  and  Sterman  (2000)  specifies  the  modeling  process  with  examples  and 
describes  numerous  applications.  System  dynamics  is  used  to  build  causal-based  (vs. 
correlation-based)  models  that  reflect  the  components  and  interactions  that  drive 
behavior  and  performance.  The  methodology  has  been  used  extensively  to  explain, 
design,  manage,  and,  thereby,  improve  the  performance  of  many  types  of  systems, 
including  development  projects.  The  system  dynamics  perspective  focuses  on  how  the 
internal  structure  of  a  system  impacts  system  and  managerial  behavior  and,  thereby, 
performance  over  time.  The  approach  is  unique  in  its  integrated  use  of  stocks  and  flows, 
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causal  feedback,  and  time  delays  to  model  and  explain  processes,  resources, 
information,  and  management  policies.  Stocks  represent  accumulations  or  backlogs  of 
work,  people,  information,  or  other  portions  of  the  system  that  change  over  time.  Flows 
represent  the  movement  of  those  commodities  into,  between,  and  out  of  stocks.  The 
methodology’s  ability  to  model  many  diverse  system  components  (e.g.,  work,  people, 
money,  value),  processes  (e.g.,  design,  technology  development,  production, 
operations,  quality  assurance),  and  managerial  decision-making  and  actions  (e.g., 
forecasting,  resource  allocation)  makes  system  dynamics  useful  for  modeling  and 
investigating  military  operations,  the  design  of  materiel,  and  acquisition. 

When  applied  to  acquisition  programs,  system  dynamics  has  focused  on  how 
performance  evolves  in  response  to  interactions  among  development  strategy  (e.g., 
evolutionary  development  versus  traditional),  managerial  decision-making  (e.g.,  scope 
developed  in  specific  blocks),  and  development  processes  (e.g.,  concurrence).  System 
dynamics  is  appropriate  for  modeling  acquisition  because  of  its  ability  to  explicitly  model 
critical  aspects  of  development  projects.  System  dynamics  models  of  development 
projects  are  purposefully  simple  relative  to  actual  practice  to  expose  the  relationships 
between  causal  structures  and  the  behavior  and  performance  that  they  create. 
Therefore,  although  many  processes  and  features  inherent  in  system  design  and  used 
by  participants  interact  to  determine  performance,  only  those  that  describe  features 
related  to  the  topic  of  study  are  included  in  system  dynamics  models. 

System  dynamics  has  been  successfully  applied  to  a  variety  of  development  and 
project  management  issues,  including  rework  (Cooper,  1993a,  1993b, 1993c;  Cooper  & 
Mullen,  1993),  the  prediction  and  discovery  of  failures  in  project  fast-track 
implementation  (Ford  &  Sterman,  2003b),  poor  schedule  performance  (Abdel-Hamid, 
1988),  tipping  point  structures  in  projects  (Taylor  &  Ford,  2008,  2006),  contingency 
management  (Ford,  2002),  resource  allocation  (Joglekar  &  Ford,  2005;  Lee,  Ford,  & 
Joglekar,  2007),  and  the  impacts  of  changes  (Rodriguez  &  Williams,  1998;  Cooper, 
1980)  and  concealing  rework  requirements  (Ford  &  Sterman,  2003a)  on  project 
performance.  See  Lyneis  and  Ford  (2007)  for  a  review  of  the  application  of  system 
dynamics  to  projects  and  project  management. 
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System  dynamics  has  also  been  applied  to  military  systems,  including  planning 
and  strategy  (Melhuish,  Pioch,  &  Seidel,  2009;  Bakken  &  Vamraak,  2003;  Duczynski, 
2000;  McLucas,  Lyell,  &  Rose,  2006),  workforce  management  (Bell  &  Liphard,  1978), 
technology  (Bakken,  2004),  command  and  control  (Bakken  &  Gilljam,  2003;  Bakken, 
Gilljam,  &  Haerem,  2004),  operations  (Bakken,  Ruud,  &  Johannessen,  2004;  Coyle  & 
Gardiner,  1991),  logistics  (Watts  &  Wolstenholme,  1990  ),  acquisition  (Ford,  Housel,  & 
Dillard,  2010;  Ford,  Housel,  &  Mun,  2011;  Ford  &  Dillard  2009a,  2009b,  2008; 
Bartolomei,  2001;  Homer  &  Somers,  1988)  and  large  system  programs  (Cooper,  1994; 
Lyneis,  Cooper,  &  Els,  2001).  Coyle  (1996)  provides  a  survey  of  applications  of  system 
dynamics  to  military  issues.  In  the  current  work,  system  dynamics  is  used  to  model  ship 
maintenance  operations  to  generate  realistic  forecasts  of  performance.  The  recent 
works  by  Ford,  Housel,  and  Mun  (2011)  and  Ford,  Housel,  and  Dillard  (2010)  are 
particularly  relevant  to  the  current  research  because  they  successfully  demonstrated  the 
ability  of  system  dynamics  to  be  integrated  with  knowledge  value  added  analysis  for 
DoD  acquisition. 
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IRM  Approach 


A.  Integrated  Risk  Management 

Integrated  risk  management  (IRM)  is  an  eight-step,  quantitative  software-based 
modeling  approach  for  the  objective  quantification  of  risk  (cost,  schedule,  technical), 
flexibility,  strategy,  and  decision  analysis.  The  method  can  be  applied  to  program 
management,  resource  portfolio  allocation,  return  on  investment  to  the  military 
(maximizing  expected  military  value  and  objective  value  quantification  of  nonrevenue 
government  projects),  analysis  of  alternatives  or  strategic  flexibility  options,  capability 
analysis,  prediction  modeling,  and  general  decision  analytics.  The  method  and  toolset 
provide  the  ability  to  consider  hundreds  of  alternatives  with  budget  and  schedule 
uncertainty,  and  provide  ways  to  help  the  decision-maker  maximize  capability  and 
readiness  at  the  lowest  cost.  This  methodology  is  particularly  amenable  to  resource 
reallocation  and  has  been  taught  and  applied  by  the  authors  for  the  past  10  years  at 
over  100  multinational  corporations  and  over  30  projects  at  the  U.S.  Department  of 
Defense  (DoD). 

IRM  provides  a  structured  approach  that  will  yield  a  rapid,  credible,  repeatable, 
scalable,  and  defensible  analysis  of  cost  savings  and  total  cost  of  ownership  while 
ensuring  that  vital  capabilities  are  not  lost  in  the  process.  The  IRM  +  KVA  methods  do 
this  by  estimating  the  value  of  a  system  or  process  in  a  common  and  objective  way 
across  various  alternatives  and  providing  the  return  on  investment  (ROI)  of  each  in 
ways  that  are  both  comparable  and  rigorous.  These  ROI  estimates  across  the  portfolio 
of  alternatives  provide  the  inputs  necessary  to  predict  the  value  of  various  options.  IRM 
incorporates  risks,  uncertainties,  budget  constraints,  implementation,  life-cycle  costs, 
reallocation  options,  and  total  ownership  costs  in  providing  a  defensible  analysis 
describing  management  options  for  the  path  forward.  This  approach  identifies  risky 
projects  and  programs,  while  projecting  immediate  and  future  cost  savings,  total  life- 
cycle  costs,  flexible  alternatives,  critical  success  factors,  strategic  options  for  optimal 
implementation  paths/decisions,  and  portfolio  optimization.  Its  employment  presents 
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ways  for  identifying  the  potential  for  cost  overruns  and  schedule  delays  and  enables 
proactive  measures  to  mitigate  those  risks.  IRM  provides  an  optimized  portfolio  of 
capability  or  implementation  options  while  maintaining  the  value  of  strategic  flexibility. 

In  the  extant  case,  IRM  provides  a  way  to  differentiate  among  various 
alternatives  for  implementation  of  3D  PDF  and  Logistics  Team  Center  with  respect  to 
ship  maintenance  processes,  and  to  postulate  where  the  greatest  benefit  could  be 
achieved  for  the  available  investment  from  within  the  portfolio  of  alternatives.  As  a 
strategy  is  formed  and  a  plan  developed  for  its  implementation,  the  toolset  provides  for 
inclusion  of  important  risk  factors,  such  as  schedule  and  technical  uncertainty,  and 
allows  for  continuous  updating  and  evaluation  by  the  program  manager  to  understand 
where  these  risks  come  into  play  and  make  informed  decisions  accordingly. 

B.  Data  Collection 

1.  Data  Collection  Methods 

Data  on  the  practices  of  Dutch  industry  and  naval  ship  maintenance  proved  very 
difficult  and  time  consuming  to  obtain.  Initial  contact  with  Dutch  industry  participants  and 
ship  maintenance  technology  providers  developed  slowly  over  several  months  into 
relationships  that  eventually  led  to  data  collection  opportunities.  Several  sources  of  data 
were  utilized,  including  a  Dutch  shipbuilder  (Damen)  and  the  Royal  Dutch  Navy  (RDN). 
Data  on  the  use  of  technology  in  Dutch  fleet  maintenance  was  collected  by  two  primary 
methods:  (1)  in-person  interviews  and  meetings  with  managers  of  the  leading 
corporation  in  the  Dutch  shipbuilding  industry  (Damen)  and  officers  and  civilian 
employees  of  the  Royal  Dutch  Navy,  and  (2)  tours  of  three  Dutch  shipbuilding  and 
maintenance  facilities. 

In-person  interviews  and  meetings  with  managers  of  Damen,  the  leading 
corporation  in  the  Dutch  shipbuilding  industry,  and  officers  and  a  civilian  employee  of 
the  Royal  Dutch  Navy  occurred  during  a  data  collection  trip  by  one  of  the  research  team 
members  (Ford)  to  the  Netherlands  in  June  2012,  as  did  the  tours  of  Dutch  ship  building 
and  maintenance  facilities.  Meetings,  semi-structured  interviews,  and  extended 
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discussions  were  held  with  six  managers  of  Damen  Industries  and  the  Royal  Dutch 
Navy  in  three  locations  over  three  days.  See  Appendix  A  for  a  list  of  informants  and  site 
visit  locations.  At  these  meetings,  Damen  managers  made  presentations  on  Damen’s 
operations,  uses  of  technologies,  investigations  of  specific  technologies  for  potential 
development  and  adoption  (including  3D  LST  and  CPLM  software),  Integrated  Logistics 
System,  and  information  technology  products  under  development  for  use  in  ship 
maintenance2.  Separately,  a  meeting  and  semi-structured  interview  was  conducted  with 
the  two  Royal  Dutch  Navy  officers  responsible  for  ship  maintenance  at  the  RDN 
shipyard  at  Nieuwe  Haven  in  Den  Helder.  Tours  of  the  Royal  Dutch  Naval  fleet 
maintenance  facility  in  Nieuwe  Haven  and  two  Damen  shipyards  were  provided  during 
the  data  collection  trip. 

2.  Data  Collection  Results — Damen’s  Use  of  Technology 

The  Damen  Shipyards  Group  (www.damen.nl/)  is  a  large  Dutch  shipbuilding  firm 
with  worldwide  operations  (1 1  shipyards  with  five  outside  The  Netherlands).  The  firm 
was  started  in  1922  by  Jan  and  Rien  Damen.  The  firm  grew  substantially  after  Kommer 
Damen  (the  current  owner)  bought  it  in  1969  and  introduced  modular  and  standardized 
shipbuilding  to  the  industry.  The  firm  now  employs  over  6,000  persons  and  builds  an 
average  of  150  vessels  per  year.  The  firm  obtained  Damen  Schelde,  which  focuses 
exclusively  on  naval  ship  design,  building,  and  maintenance  relatively  recently  (in  2000). 
Damen  Schelde  manufactures  an  average  of  one  to  two  ships  per  year,  employs  about 
550  people,  and  performs  about  €210  million  per  year.  Damen  Schelde  acts  as  the 
prime  contractor  and  integrator  on  its  shipbuilding  projects,  utilizing  many 
subcontractors.  Although  Damen  Schelde  provides  ship  maintenance  services  to  its 
international  (i.e.,  not  Dutch)  customers,  it  does  not  provide  any  ship  maintenance 
services  for  the  Royal  Dutch  Navy. 


2  Copies  of  presentations  were  requested,  but  not  provided.  Data  collection  results  are  based  on  notes 
taken  by  the  investigator  during  the  meetings,  interviews,  and  tours  of  facilities. 
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Damen  Schelde  has  used  an  Integrated  Logistics  System  (ILS)  since  2002  to 
manage  the  shipbuilding  process  from  project  initiation  through  the  development  of  a 
logistics  plan  for  customers.  The  ILS  is  the  plan  for  the  development  of  a  ship  and 
includes  ship  design,  production,  QAQC  (quality  assurance,  quality  control),  training  of 
ship  operators,  and  coordination  with  customers.  The  ILS  does  not  include  service 
contracts  or  life-cycle  costs  due  to  the  difficulty  of  forecasting  those  costs.  The  focus  of 
the  ILS  is  to  provide  maximum  ship  operational  availability,  reliability,  and 
maintainability.  It  does  this  partially  by  using  a  single  point  of  contact  within  Damen 
throughout  the  project  who  manages  an  interdisciplinary  team  (e.g.,  engineering,  work 
preparation,  procurement,  service).  Damen  Schelde  currently  uses  a  variety  of 
information  technologies  to  facilitate  their  ILS  approach  to  shipbuilding  and  is  constantly 
investigating  new  technologies  that  may  improve  their  design  and  manufacturing.  Of 
particular  relevance  to  the  current  work,  Damen  Schelde  uses  four  separate  software 
products  to  manage  their  shipbuilding:  an  advanced  3-dimensional  CADD  program  for 
design,  a  CPLM  product  as  a  database  for  ship  components3,  an  Enterprise  Resource 
Planning  (ERP)  system,  and  a  software  tool  for  scheduling.  The  latter  three  of  these 
systems  are  connected  to  users  with  a  project  information  portal  developed  by  Damen 
Schelde.  The  informant  reported  that  Damen  developed  the  portal  because  the  CPLM 
product  did  not  include  adequate  user  interfaces. 

Damen  Schelde  has  investigated  and  is  currently  investigating  other  technologies 
for  potential  adoption.  Four  technologies  were  described  and  discussed: 

1 .  3D  Laser  Scanning  Technology  (3D  LST):  This  technology  was 

investigated,  but  was  assessed  to  currently  be  too  immature  for  adoption 
by  Damen  Schelde.  The  investigation  included  a  discussion  of  the  current 
use  of  the  technology  in  the  automobile  industry,  as  well  as  its  potential 
use  to  scan  engine  rooms  and  for  floor  flattening.  The  use  of  360  degree 
photography  (often  used  in  conjunction  with  3D  LST)  was  considered  by 
Damen  Schelde  as  a  potential  tool  for  training.  See  Komoroski  (2005)  for 
more  details  on  3D  LST. 


3  Damen  Schelde  did  not  purchase  the  integrated  logistic  package  available  from  Siemens. 
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2. 


3D  PDF  files:  3-dimensional  animated  “movies”  of  shipbuilding  can  be 
created  in  a  PDF  format  (by  Adobe  Acrobat®)  and  sent  to  shipyards  for 
use  in  the  field  by  craftsmen  who  view  the  file  on  an  electronic  reader 
(e.g.,  iPad®).  The  files  would  replace  flat  drawings  for  use  in  construction. 
The  file  visually  communicates  the  sequence  of  building  (or  maintenance) 
operations  and  components  and  operations  can  have  notes  attached  to 
them  that  provide  additional  information  (e.g.,  part  numbers,  warnings  of 
special  issues).  The  ability  to  animate  these  files  allows  engineers  to 
visually  show  craftsmen  sequences  of  operations,  routes  of  access  and 
egress  for  Line  Replaceable  Units  (LRU4),  and  other  information  that  is 
difficult  or  impossible  to  show  with  traditional  static  2-dimensional 
drawings.  The  use  of  this  technology  shifts  the  understanding  of  the 
design  intention  from  the  designers  (in  the  Netherlands)  to  the  shipbuilding 
yard  (typically  in  other  countries  around  the  world).  The  use  of  visual 
information  (the  animation  of  steps)  is  expected  to  greatly  improve 
communication  across  languages,  since  many  of  the  craftsmen  in 
Damen’s  shipyards  do  not  read  English  well.  Damen  considers 
improvements  in  information  content  communicated  to  be  the  primary 
benefit  of  this  system  (versus  cost  savings).  Damen  Schelde  is  very 
optimistic  about  the  potential  for  this  technology  to  improve  its  operations 
and  is  actively  working  on  developing  it  (e.g.,  selecting  software, 
addressing  the  importing  of  the  3D  design  drawings).  Generating  the 
animated  files  and  adding  the  building  steps  to  the  design  files  is  expected 
to  be  relatively  easy  once  the  system  has  been  developed. 

3.  SIGMA  Shipbuilding  Strategy:  This  is  a  standardized  process  for  creating 
a  ship  that  spans  from  design  through  materials  procurement,  production, 
and  testing  of  a  ship.  The  key  feature  of  the  strategy  is  the  use  of  modular 
ship  sizes  and  systems  that  can  be  easily  adapted  to  specific  customer 
needs.  For  example,  Damen  Schelde  has  disaggregated  an  entire  ship 
into  five  standardized  modules  (e.g.,  fore,  midship,  aft)  with  major  systems 
located  in  specific  sections.  Each  module  is  considered  a  subproject.  As 
an  example  of  an  advantage  provided  by  the  strategy,  the  modules  and 
their  interfaces  are  designed  such  that  the  ship  can  be  made  longer  by 
adding  an  additional  midsection.5 

4.  Radio  Frequency  Identification  (RFID):  This  established  technology  is 
being  considered  for  use  to  improve  Damen’s  supply  chain  management. 
Primary  benefits  are  believed  to  be  improved  value  of  information  and  a 
reduction  in  durations  for  getting  information  into  Damen  databases  (e.g., 


4  Line  Replaceable  Unit  is  a  commonly  used  term  in  manufactured  devices  for  any  modular  component 
that  is  designed  to  be  interchangeable. 

5  This  portion  of  the  SIGMA  strategy  applies  the  Boeing  strategy  for  the  design  and  production  of  the  737 
that  has  different  lengths  to  shipbuilding. 
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warehouse  contents,  components  on  specific  ships).  Both  passive  and 
active  tags  are  being  considered. 

Damen  Services  also  develops  advanced  technologies  for  use  by  Damen 
Enterprises.  Damen  Services  focuses  on  providing  ongoing  maintenance  parts  and 
services  to  Damen  customers  after  a  ship  has  been  designed,  built,  and  delivered,  but 
also  provides  other  services  such  as  civil  works  (e.g.,  wharves  and  storage  facilities). 

The  Maintenance  and  Spares  department  maintains  information  on  ship 
configuration  (using  an  ERP  system),  parts  inventories,  and  spare  parts  packages, 
maintenance  management  systems.  It  also  provides  information  technology  support  for 
Damen.  Damen  Services  has  grown  rapidly,  from  50  employees  in  2008  to  250 
employees  in  2012.  Their  primary  objective  for  customers  is  to  reduce  costs  and 
increase  operational  availability.  They  are  developing  a  web  portal  for  clients  that  will 
allow  clients  access  to  Damen-held  data  on  each  of  the  customer’s  ships  down  to  the 
individual  component  level.  This  will  partially  be  accomplished  with  a  work  breakdown 
system  (WBS)  that  disaggregates  a  ship  or  system  into  product  parts  (e.g.,  engine,  bilge 
pump)  and  a  functional  breakdown  system  that  disaggregates  the  ship  into  functions 
(e.g.,  port  propulsion)  that  are  met  with  a  product  part  (in  the  WBS)  and  have  an 
associated  maintenance  schedule,  which  includes  monitoring  measurements  and 
frequency,  parts  documentation,  and  so  forth.  The  WBS  has  three  levels:  Subsystems 
(e.g.,  propulsion,  hoisting)  with  a  typical  ship  having  20-70  subsystems,  Level  2  Parts 
(e.g.,  pump,  shaft)  with  about  1 ,000  per  ship,  and  Level  3  Parts  (e.g.,  bolt,  flange)  with 
70,000-80,000  per  ship. 

This  system  will  be  linked  with  an  online  parts  ordering  portal  so  that  customers 
can  order  parts  from  Damen  (similar  to  Amazon’s  online  selling  of  books,  etc.).  Damen 
Services  plans  to  use  the  information  captured  through  this  system  (e.g.,  frequency  of 
ordering  of  specific  components)  to  develop  maintenance  optimization  information. 
Damen  Services  envisions  three  types  of  maintenance:  corrective  maintenance  (after 
the  component  needs  work),  preventative  maintenance  (based  on  forecasts  of 
maintenance  needs),  and  condition-based  maintenance  (based  on  actual  conditions  of 
components).  Condition-based  maintenance  is  an  optimized  version  of  preventative- 
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based  maintenance  that  is  currently  under  development.  It  requires  sensors  to  collect 
data  on  component  conditions  that  will  be  used  to  generate  condition  assessments. 

3.  Data  Collection  Results — Royal  Dutch  Navy  (RDN)  Fleet  Maintenance 

Data  collection  directly  from  the  RDN  was  particularly  valuable  for  at  least  two 
reasons.  First,  as  the  navy  of  a  sovereign  country  with  objectives  that  are  similar  to 
those  of  the  United  States,  the  objectives  and  issues  of  the  RDN  are  more  likely  to 
match  those  of  the  U.S.  Navy  than  those  of  some  other  nations.  Data  collection 
supported  this  assumption.  For  example,  technology  leadership,  interoperability,  and 
reliability  in  meeting  operational  needs  are  paramount  to  the  RDN,  and  the  RDN  has 
recently  experienced,  and  expects  to  continue  to  experience,  reductions  in  budgets  just 
as  is  the  case  with  the  U.S.  Navy.  The  Dutch  navy  continues  to  face  budget  cuts  and 
increasing  technology  needs,  is  currently  in  reorganization  to  reduce  total  workforce 
(internal  to  the  navy  and  civilian  naval  workforce)  by  20%,  and  is  transferring  from  their 
legacy  information  systems  to  an  integrated  ERP  system  for  maintenance  operations. 
Second,  the  RDN  performs  all  of  the  maintenance  on  its  fleet,  thereby  making  it  the 
primary  data  source  concerning  RDN  fleet  maintenance  process  performance. 

The  interviews  with  the  two  RDN  officers  in  the  Naval  Maintenance  and  Service 
Agency  provided  a  general  introduction  to  the  issues  faced  by  the  Dutch  navy  in 
building  and  maintaining  its  fleet.  The  RDN  addresses  its  challenges  by  means  similar 
to  those  used  by  the  U.S.  Navy,  such  as  waiting  for  technology  to  mature  (technology 
readiness  level  [TRL]  >  7  before  adoption)  and  incremental  capability  increases  based 
on  budgets.  Noticeably  different,  both  the  RDN  and  Damen  described  the  critical  role 
and  standard  Dutch  practice  of  adjusting  requirements  to  meet  budgets  in  shipbuilding. 
The  RDN  is  facing  increasing  pressure  to  control  life-cycle  costs  in  its  fleet,  which  are 
largely  driven  by  personnel  and  fuel.  This  has  led  them  to  approve  significantly  stricter 
operations  manning  requirements  for  ship  design  (i.e. ,  lower  maximum  shipboard 
personnel),  which  has  driven  Damen  to  increase  the  use  of  automation  in  their  ship 
designs. 
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The  primary  informant  on  RDN  fleet  maintenance  operations  provided  a  diagram 
of  those  operations  (Figure  4)  and  a  written  description  of  each  of  the  steps  identified  in 
the  diagram. 


Logistics  Process  Royal  Netherlands  Navy 


Naval  Maintenance 
Establishment 


Figure  4.  Diagram  of  Royal  Dutch  Navy  Fleet  Maintenance  Processes 

(Kense,  2012) 

The  process  steps  shown  in  Figure  4  were  described  in  writing  by  the  informant 
with  the  following  list.6  In  the  list,  the  abbreviation  “LRU”  stands  for  “Line  Replaceable 
Unit,”  a  commonly  used  term  in  the  area  of  manufactured  devices  for  any  modular 
component  that  is  designed  to  be  interchangeable.  MIL-PRF-49506,  Notice  1  of  18 


6  Process  step  descriptions  have  been  transcribed  exactly  as  provided  in  English  by  the  RDN,  including 
uncommon  English  grammar  and  spelling. 
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(USAMC  Logistics  Support  Activity,  1996),  “Performance  Specification  for  Logistics 
Management  Information,”  provides  the  following  definition: 

An  LRU  is  an  essential  support  item  which  is  removed  and  replaced  at  the  field 
level  to  restore  the  end  item  to  an  operational  ready  condition.  Conversely,  a 
non-LRU  is  a  part,  component,  or  assembly  used  in  the  repair  of  an  LRU,  when 
the  LRU  has  failed  and  has  been  removed  from  the  end  item  for  repair. 

C.  Logistic  Process  Royal  Netherlands  Navy 

In  case  LRU  fails  the  on-board  personnel  will  replace  this  LRU  by  a  spare  (on-board; 
OLM  qualification  required). 

The  defect  LRU  will  be  send  to  the  warehouse,  and  a  “new”  LRU  will  be  send  to  the 
ship. 

The  defect  LRU  will  be  send  to  the  Naval  Maintenance  Establishment  (NME)  for  repair. 
After  the  LRU  is  repaired  it  will  be  send  to  the  warehouse  again  “as  good  as  new”  (DLM 
qualification  required). 

If  the  NME  needs  parts  to  repair  an  LRU,  the  parts  will  be  extracted  from  the  industry, 
when  the  NME  is  not  able  to  repair  this  LRU,  it  can  be  send  to  the  manufacturer.  Also, 
manpower  can  be  hired  to  fix  problems. 

If  spare  is  not  available,  sometimes  it  will  be  cannibalized  from  another  ship. 

If  the  on-board  personnel  is  not  able  to  fix  the  problem  by  themselves  (due  to  the 
complexity  of  the  failure)  assistance  from  the  NME  is  needed  (ILM  qualification 
required). 

If  the  problem  is  too  complex  for  the  NME  also,  the  industry  can  be  hired  to  solve  this 
problem. 

The  following  seven  process  steps  were  elaborated  on  by  the  informant: 

Step  1 :  Performed  onboard,  for  example  to  provide  operational 
maintenance  of  weapons  systems 

Step  2:  Purely  a  transit  operation  that  requires  only  a  truck  driver  (if  ship  is 
in  port) 

Step  4:  Requires  DLM  level  of  training 
Step  5:  Requires  OLM  level  of  training 
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Step  6:  Requires  ILM  level  of  training  (=  LTS  +  MTS  +  10-25  days  of 
training) 

Step  7:  Requires  DLM  level  of  training 

The  abbreviations  DLM,  OLM,  and  ILM  refer  to  Dutch  terms  for  training  levels. 
Fleet  maintenance  for  the  RDN  requires  a  minimum  of  completion  of  education  at  a 
Lower  Technical  School  (LTS)  and  a  Middle/Intermediate  Technical  School  (MTS).  The 
Lower  Technical  School  is  typically  attended  between  ages  12-16  and  the  Middle 
Technical  School  is  typically  attended  between  ages  16-21 .  After  the  completion  of  LTS 
and  MTS,  future  RDN  ship  fleet  maintenance  personnel  must  complete  at  least  one  of 
three  other  forms  of  training: 

•  OLM — 5-1 0  days  of  training 

•  ILM — 1 0-25  days  of  training 

•  DLM — 1 5-35  days  of  training 

Manufacturer  training  can  take  either  of  two  alternative  training  paths: 

•  LTS  then  MTS  then  either  OLM  or  ILM  or  DLM 

•  LTS  then  MTS  then  OLM  then  ILM  then  DLM 

The  information  presented  here  was  augmented  by  a  tour  of  the  Naval 
Maintenance  Establishment  (NME)  maintenance  and  repair  facilities.  The  NME  provides 
essentially  all  maintenance  and  repair  for  the  RDN  fleet  and  the  NME  facilities  can,  and 
do,  perform  the  required  work  on  RDN  ship  components.  This  requires  a 
comprehensive  set  of  equipment  and  skilled  personnel  that  cover  the  wide  range  of 
materials  and  components.  Examples  of  testing,  maintenance,  and  repair  capabilities 
seen  on  the  tour  include,  but  are  not  limited  to,  the  repair  of  a  wide  variety  of  weapons 
systems,  radar  systems  testing  and  repair,  design  and  manufacturing  of  printed  circuit 
boards,  and  the  manufacturing  of  optical  lens  for  submarine  periscopes.  NME  holds 
220,000  total  items  in  the  warehouse  valued  at  about  €500  million.  An  average  Dutch 
naval  ship  contains  about  60,000  components. 
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D.  System  Dynamics  Model  Structure 


The  system  dynamics  model  simulates  the  movement  of  LRU  among  the  various 
locations  where  they  are  used,  stored,  or  repaired.  These  accumulations  are  referred  to 
as  stocks  (Sterman,  2000).  Each  flow  of  LRUs  between  two  stocks  represents  the 
processing  rate  of  one  of  the  process  steps  in  a  knowledge  value  added  model.  A 
simplified  diagram  of  the  stocks  and  flows  of  the  model  are  shown  in  Figure  5.  Boxes 
represent  stocks,  or  accumulations  of  LRU.  Each  stock  in  Figure  5  represents  a  location 
in  Figure  4,  plus  on-board  LRU  storage  as  a  separate  LRU  accumulation.  Arrows  with 
valve  symbols  in  Figure  5  represent  the  movement  of  LRUs  between  stocks.  Numbers 
in  parenthesis  in  the  titles  of  flows  represent  the  process  steps  shown  in  Figure  4  (ovals 
with  arrows)  and  the  KVA  model  process  steps  (described  later). 


NME  (4) 


Figure  5.  Royal  Dutch  Navy  Ship  Maintenance:  Stocks  and  Flows 

of  the  System  Dynamics  Model 

The  sizes  of  the  flows  in  the  system  dynamics  model  describe  the  rate  of 
movement  of  LRUs  among  the  stocks.  Therefore,  the  simulated  flows  in  the  system 
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dynamics  model  become  direct  inputs  to  the  “times  processed  per  year”  portion  of  the 
KVA  models.  Flow  rates  were  modeled  to  reflect  the  sequence  of  processes  in 
operations.  For  example,  in  normal  operations,  the  replacement  of  a  broken  LRU  in  an 
operating  ship  with  one  from  the  ship’s  on-board  storage  (“Replace  broken  LRU  from 
storage  [1]”  on  left  of  Figure  5)  would  be  followed  by  the  broken  LRU  in  storage  being 
replaced  by  an  operational  LRU  from  the  warehouse  (“Replace  broken  LRU  from 
warehouse  on  onboard  storage  [8]”  at  top  in  Figure  5).  This  replacement  would  be 
followed  by  the  broken  LRU  being  sent  to  the  NME  where  it  would  be  repaired  and 
returned  to  the  warehouse  (“NME  repairs  broken  LRU  in  warehouse  [3]”  on  right  in 
Figure  5).  These  precedencies  are  modeled  by  having  the  downstream  process  equal  to 
its  preceding  process  step  with  a  delay  that  reflects  the  transit  and  subsequent 
processing  time.  Some  flows  (e.g.,  NME  repairs  broken  LRU  from  warehouse  [3])  are 
aggregations  of  multiple  upstream  flows.  Core  flows  are  based  on  the  mean  time 
between  failure  of  LRUs  and  the  fraction  of  failures  addressed  with  each  process. 

The  system  dynamics  model  was  calibrated  to  reflect  RDN  ship  maintenance. 
Quantitative  information  on  the  volume  of  process  steps  performed  in  the  maintenance 
of  the  RDN  fleet  was  requested  but  was  not  available,  primarily  due  to  the  extreme 
diversity  of  components  and  maintenance  requirements.  One  RDN  informant  described 
the  frequency  of  the  maintenance  operations  (i.e.,  Steps  1-7  presented  previously)  as 
“continuous”  and  said  that  frequency  estimates  were  very  difficult  because  of  the 
extreme  range  of  frequencies  across  component  types.  As  an  example,  the  informant 
said  that  work  on  some  components  happens  daily  while  work  on  other  types  of 
components  happens  only  once  every  few  years.  The  informant  provided  the  example 
that  when  a  warship  was  at  sea  for  30  days  only  process  Step  1  (on-board  repairs) 
would  occur,  but  if  the  ship  were  at  port  other  process  steps  might  be  used.  Therefore, 
the  modeler  based  calibration  for  a  portion  of  the  system  dynamics  model  on  publicly 
available  data,  data  collected  (e.g.,  numbers  of  LRU  in  NME  and  onboard  a  typical 
ship),  and  estimated  conditions  of  peacetime  operations  near  Dutch  ports.  Publically 
available  data  included  the  types  and  numbers  of  ships  in  the  Dutch  navy  (’’List  of 
Ships,”  2012;  Table  2). 
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Table  2.  Royal  Dutch  Navy  Ship  Types  and  Number 


Ship  Type 

Number 

Frigate 

12 

Landing  Platform 

2 

Replenishment 

2 

Submarine 

4 

Mine  detection 

6 

Dive  support 

4 

Hydrographical  survey 

2 

Training 

2 

Tugs  -  large 

5 

Tugs  -  harbor 

7 

Landing  craft 

17 

Patrol  boat  -  off  shore 

4 

Patrol  boat  -  in  shore 

6 

Cutter 

3 

TOTAL 

76 

Calibration  estimates  were  made  using  this  information  as  follows.  Data  not 
documented  in  Table  2  are  modeler  estimates. 

Total  LRU  in  use  on  all  ships  =  60k  LRU/ship  *  76  ships  =  4,560k  LRU  on 
ships 

Assuming  one  ship  of  each  of  the  14  ship  types  is  considered  “sacrificial”  and 
used  for  cannibalization,  then 

Total  LRU  in  use  on  the  62  (=  76-14)  “operating”  ships  =  62  ships  *  60k 
LRU/ship  =  3,720k  LRU 

Total  LRU  in  use  on  the  14  “sacrificial”  ships  =  4,560k  -  3,720k  =  840k 
LRU 

In  addition,  each  ship  keeps  25%  of  its  LRU  in  on-board  ship  storage: 

62  ships  *  (25%)(60k  LRU/ship)  =  930k  LRU  in  storage  on  operating  ships 

14  ships  *  (25%)(60k  LRU/ship)  =  210k  LRU  in  storage  on  sacrificial  ships 

Total  LRU  on  sacrificial  ships  =  840k  +  210k  =  1050k  LRU  on  sacrificial 
ships 
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The  warehouse  initially  holds  one  complete  set  for  each  vessel  type: 


60k  LRU/ship  *  14  ship  types  =  840k  LRU 

The  number  of  LRU  at  NME  is  only  the  LRU  that  are  currently  being  repaired  by 
NME  (i.e.,  all  LRU  storage  occurs  at  the  warehouse  and  none  at  NME  [consistent  with 
researcher  observations]). 

The  following  fractions  of  broken  LRUs  are  addressed  with  each  solution: 

25%  of  broken  LRU  are  replaced  with  on-board  replacements7 

10%  of  broken  LRU  are  cannibalized  from  other  ships8 

35%  of  broken  LRU  are  replaced  with  LRU  in  warehouse9 

25%  of  broken  LRU  are  repaired  by  NME  directly  without  passing  through 
warehouse 

5%  of  broken  LRU  are  repaired  directly  by  industry 

100%  TOTAL 

The  estimates  assume  that  15%  of  LRU  repaired  directly  by  NME  need 
assistance  from  industry. 

E.  KVA  Models  to  the  Royal  Dutch  Navy  Ship  Maintenance 

Four  knowledge  value  added  models  were  built  based  on  the  Royal  Dutch  Navy 
Ship  Maintenance  processes: 

1.  Baseline  RDN  ship  maintenance  processes 

2.  Baseline  RDN  ship  maintenance  processes  changed  to  reflect  the 
adoption  and  use  of  a  logistics  package  from  an  integrated  CPLM  system 
such  as  was  investigated  by  Damen 


7  These  LRU  are  then  sent  to  the  warehouse  and  replaced  with  an  operational  LRU  from  the  warehouse. 

8  These  LRU  are  then  sent  to  the  warehouse  and  replaced  with  an  operational  LRU  from  the  warehouse. 

9 These  LRU  are  then  sent  to  the  NME  for  repair  and  returned  to  the  warehouse. 
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3.  Baseline  RDN  ship  maintenance  processes  changed  to  reflect  the 
adoption  and  use  of  3D  PDF  modeling  managed  with  a  CPLM  system  as 
planned  by  Damen 

4.  Baseline  RDN  ship  maintenance  processes  changed  to  reflect  the 
adoption  and  use  of  a  logistics  package  and  3D  PDF  modeling  managed 
by  an  integrated  CPLM  system 

Inputs  to  these  models  were  generated  as  follows: 

Process  Descriptions — The  seven  basic  process  steps  used  by  the  RDN  to 
maintain  the  fleet  were  taken  from  data  collected  from  the  RDN  (Figure  4)  and 
descriptions  provided  by  the  manager  of  the  NME.  Two  additional  process  steps  (8  and 
9)  were  added  based  on  the  logic  that  broken  LRU  in  onboard  storage  or  cannibalized 
ships  would  be  replaced  with  operating  LRU  from  the  warehouse. 

Title  of  Head  Process  Executer — The  KVA  modeler  matched  the  levels  and 
types  of  training  received  in  the  different  levels  of  training  as  described  by  the 
informants  to  the  process  steps  based  on  process  step  requirements. 

Number  of  Employees — The  KVA  modeler’s  estimate  based  on  manpower 
requirements  to  perform  each  process  step  in  the  maintenance  of  pumps  scenario. 

Corresponding  Pay  Grades — The  KVA  modeler’s  estimate  of  relative  hourly 
pay  rates  for  skill  levels  described  by  training  requirements.  Estimated  values  include 
labor  burden  and  overhead. 

Rank  Order  of  Difficulty — The  KVA  modeler’s  estimate  based  on  understanding 
of  processes  from  informants. 

Actual  Average  Training  Period — Based  on  data  provided  by  informants  (see 
previous  data  description). 

Percentage  Automation — The  KVA  modeler’s  estimate  in  the  base  case  based 
on  understanding  of  processes  from  informants.  Modeler’s  estimate  of  changes  due  to 
technology  adoptions  based  on  previous  KVA  models  of  ship  maintenance  processes. 
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Times  performed  in  a  Year — Output  from  the  system  dynamics  model. 


Average  Time  to  Complete — The  KVA  modeler’s  estimate  for  the  base  case 
based  on  understanding  of  processes  from  informants.  The  modeler’s  estimate  of 
changes  due  to  technology  adoptions  for  other  KVA  models. 

Automation  Tools — The  KVA  modeler’s  estimate  for  the  base  case  based  on 
understanding  of  processes  from  informants.  The  modeler’s  estimate  of  changes  due  to 
technology  adoptions  for  other  KVA  models. 

F.  Model  Simulations  and  Results 

The  system  dynamics  model  was  simulated  to  represent  the  four  technology 
adoption  scenarios  described  in  the  previous  section.  The  output  of  each  system 
dynamics  model  simulation  was  used  as  input  to  a  KVA  model.  Those  KVA  models 
were  then  used  to  estimate  the  return  on  investment  (ROI)  of  each  process  in  each  of 
the  four  scenarios  and  the  cumulative  ROI  for  each  scenario.  The  results  based  on  the 
models  and  their  calibrations  described  previously  are  shown  in  Table  3. 
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Table  3.  Knowledge  Value  Added  Model  Results 


Return  On  Investment  (ROI) 

Process 

Description 

Baseline 

Add 

Logistics 

Add  3D 

PDF 

Add 

Logistics 
&  3D  PDF 

1 

Replace  LRU  with  on-board 
spare 

90% 

261% 

501% 

464% 

2 

Replace  operating  LRU  with 
warehouse  spare 

90% 

151% 

621% 

1 027% 

3 

NME  repairs  warehouse  LRU 
and  returns  it  to  warehouse 

8% 

65% 

95% 

236% 

4 

Manufacturer  repairs  LRU  for 
NME  &  it  returns  to  warehouse 

31% 

88% 

168% 

168% 

5 

Replace  on-board  LRU  with 

LRU  cannibalized  from  another 
ship 

90% 

151% 

621% 

1 027% 

6 

NME  repairs  on-board  LRU  and 
returns  it  to  ship 

265% 

10% 

99% 

192% 

7 

Industry  repairs  on-board  LRU 
and  returns  it  to  ship 

34% 

1 78% 

135% 

318% 

8 

Replace  on-board  storage  LRU 
with  warehouse  spare  (transit 
only) 

301% 

759% 

759% 

759% 

9 

Replace  cannabalized  LRU  with 
warehouse  spare  (transit  only) 

140% 

329% 

862% 

1102% 

TOTAL  ALL  PROCESSES 

35% 

77% 

135% 

274% 

Although  increased  throughput  due  to  reduced  processing  durations  (which 
increase  the  ROI  numerator)  can  partially  explain  differences  in  the  ROI  in  Table  3,  cost 
reduction  (which  decreases  the  ROI  denominator)  is  the  primary  driver  of  increases  in 
ROI.  For  example,  processes  8  and  9  are  benefitted  by  reductions  in  rework  (e.g., 
errors  in  transporting  LRU)  due  to  the  adoption  of  a  logistics  package.  This  reduces  the 
number  of  transport  trips  required  (the  function  of  these  processes),  thereby  significantly 
reducing  costs  and  increasing  the  ROI.  In  contrast,  processes  3,  4,  and  6  are  highly 
skilled  processes  that  are  difficult  to  replace  with  technology  and,  therefore,  benefit  less 
from  technology  adoption  than  other  processes.  This  results  in  a  smaller  increase  in 
ROI  for  these  processes. 
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G.  Analysis  of  Simulation  Model  Results 


A  variance  analysis  was  performed  on  the  KVA  model  results  (Table  3)  to 
evaluate  the  relative  impacts  of  the  adoption  of  different  technologies  (Table  4).  Returns 
on  investment  for  each  of  the  three  technology  adoption  alternatives  were  compared 
with  the  baseline  returns  on  investment  to  estimate  improvement  due  to  technologies 
(left  three  columns  of  results,  Table  4).  In  addition,  the  improvement  from  adopting  both 
technologies  over  adopting  only  the  3D  PDF  technology  was  estimated  (right  column, 
Table  4) 


Table  4.  Variance  Analysis  of  KVA  Model  Results 


Return  On  Investment  (ROI) 

Process 

Description 

Add  Logistics  - 
Improvement 
over  Baseline 

Add  3Dpdf  - 
Improvement 
over  Baseline 

Add  Logistics 
&  3Dpdf  - 
Improvement 
over  Baseline 

Add  Logistics  & 
3Dpdf  - 
Improvement 
over  adding  only 
3Dpdf 

1 

Replace  LRU  with  on-board 
spare 

171% 

411% 

374% 

-38% 

2 

Replace  operating  LRU  with 
warehouse  spare 

61% 

532% 

937% 

406% 

3 

NME  repairs  warehouse  LRU 
and  returns  it  to  warehouse 

57% 

87% 

227% 

140% 

4 

Manufacturer  repairs  LRU  for 
NME  &  it  returns  to  warehouse 

57% 

138% 

138% 

0% 

5 

Replace  on-board  LRU  with 

LRU  cannibalized  from  another 
ship 

61% 

532% 

937% 

406% 

6 

NME  repairs  on-board  LRU  and 
returns  it  to  ship 

-256% 

-166% 

-73% 

93% 

7 

Industry  repairs  on-board  LRU 
and  returns  it  to  ship 

145% 

101% 

284% 

183% 

8 

Replace  on-board  storage  LRU 
with  warehouse  spare  (transit 
only) 

458% 

458% 

458% 

0% 

9 

Replace  cannabalized  LRU  with 
warehouse  spare  (transit  only) 

189% 

721% 

962% 

240% 

TOTAL  ALL  PROCESSES 

42% 

100% 

239% 

139% 

Referring  to  Table  4,  adding  either  or  both  of  the  technologies  improves  overall 
ship  maintenance  ROI,  as  indicated  by  the  positive  numbers  in  the  last  row  of  Table  4. 
Adopting  3D  PDF  alone  improves  ROI  significantly  more  than  adopting  a  logistics 
package  alone  (100%  improvement  >  46%  improvement)  and  adding  both  technologies 
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improves  ROI  more  than  adding  either  technology  alone  (239%  improvement  >  42% 
improvement  or  100%  improvement),  suggesting  that  there  may  be  synergy  between 
the  technologies.  This  is  also  supported  by  the  139%  improvement  by  adding  logistics  if 
3D  PDF  is  already  in  place  (lower  right  result  in  Table  4). 

Adopting  the  technologies  does  not  impact  the  ROI  of  individual  processes 
equally.  Among  the  seven  core  processes(1-7)10  adding  only  a  logistics  package  (left 
column  of  results  in  Table  4)  increases  the  “Replace  LRU  with  on-board  spare”  (process 
1)  most,  by  171%,  and  decreases  the  return  of  process  6,  “NME  repairs  on-board  LRU 
and  returns  it  to  ship”  by  256%.  Among  the  seven  core  processes,  adding  only  3D  PDF 
increases  processes  2  and  5,  “Replace  operating  LRU  with  warehouse  spare”  and 
“Replace  on-board  LRU  with  LRU  cannibalized  from  another  shop”  most,  by  532%,  and 
decreases  the  return  of  process  6,  “NME  repairs  on-board  LRU  and  returns  it  to  ship”  by 
166%.  Among  the  seven  core  processes,  adding  both  technologies  increases 
processes  2  and  5,  “Replace  operating  LRU  with  warehouse  spare”  and  “Replace  on¬ 
board  LRU  with  LRU  cannibalized  from  another  shop”  most,  by  937%,  and  decreases 
the  return  of  process  6,  “NME  repairs  on-board  LRU  and  returns  it  to  ship,”  by  73%. 

H.  Comparison  of  Royal  Dutch  Navy  and  U.S.  Navy  Scenarios 

Previous  research  using  the  KVA  approach  developed  estimates  of  returns  on 
technology  investment  of  a  scenario  in  which  the  U.S.  Navy  adopts  3D  laser  scanning 
technology  (3D  LST)  and  collaborative  product  lifecycle  management  (CPLM)  tools  into 
the  SHIPMAIN  program.  Komoroski  (2005)  investigated  the  early  phases  of  SHIPMAIN. 
The  relevant  results  are  shown  in  Table  5. 


10  Process  8,  “Replace  on-board  storage  LRU  with  warehouse  spare  (transit  only)”  supports  process  1, 
“Replace  LRU  with  on-board  spare.”  Therefore  process  1  is  the  core  process.  Similarly,  process  9, 
“Replace  cannibalized  LRU  with  warehouse  spare  (transit  only)”  supports  process  5,  “Replace  on-board 
LRU  with  LRU  cannibalized  from  another  ship.”  Therefore  process  5  is  the  core  process. 
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Table  5. 


Preparation  for  Maintenance  Processes— As-ls  and  Radical  ROI  Differences 


Core 

Process 

Process  Title 

"AS_IS" 

ROI 

"RADICAL" 

ROI 

"RADICAL" 
improvement 
over  "AS-IS" 

1 

ISSUE  TASKING 

-59% 

-59% 

0% 

2 

INTERPRET  ORDERS 

-73% 

746% 

819% 

3 

PLAN  FOR  SHIPCHECK 

-99% 

-95% 

4% 

4 

CONDUCT  SHIPCHECK 

-74% 

1653% 

1727% 

5 

REPORT  ASSEMBLY 

-39% 

1032% 

1071% 

6 

REVISE  SCHEDULE 

12% 

882% 

870% 

7 

GENERATE  DRAWINGS 

-54% 

2977% 

3031% 

TOTALS 

-27% 

2019% 

2045% 

Note.  This  table  is  based  on  Komoroski  (2005). 

Referring  to  Table  5,  adding  the  3D  LST  and  CPLM  technologies  improves 
overall  preparation  for  maintenance  process  ROI,  as  indicated  by  the  positive  number  in 
the  lower  right  corner  of  Table  5.  Adding  these  technologies  generally  improves 
individual  processes  as  well,  as  indicated  by  the  non-negative  (and  positive  with  one 
exception)  numbers  in  the  right  column  of  Table  5.  The  range  of  improvements  across 
individual  processes  is  large,  varying  from  0%  (Issue  Tasking)  to  3031%  (Generate 
Drawings).  Cost  reduction  explains  these  differences.  For  example,  the  adoption  of 
technology  in  Core  Processes  4  (Conduct  Shipcheck)  and  7  (Generate  Drawings) 
significantly  reduce  the  number  of  people  required  to  survey  ship  conditions  (4)  or  draft 
3D  drawings  from  the  survey  data  (9),  resulting  in  large  ROI  if  the  technology  is 
adopted. 

Seaman,  Housel,  and  Mun  (2007)  used  KVA  to  model  the  later  phases  of 
SHIPMAIN.  The  relevant  results  are  shown  in  Table  6. 
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Table  6.  Maintenance  Implementation  Processes — As-ls  and  To-Be 

ROI  Comparison 

(Seaman,  Housel,  &  Mun,  2007) 


Core 

Process 

Process  Title 

Annual  As-ls 
Cost 

Annual  As-ls 
Benefits 

Annual  To-Be 
Cost 

Annual  To-Be 
Benefits 

As-ls 

ROI 

To-Be 

ROI 

Blade  ZE« 

Mlbaria  tsw  Uttar  af 

*  ^laa  =Hi'P : GAM* its 

WMIM  1 

mzjhlsiz 

1 

nijifjf^i  i 

1 

1 

mi  i 

Ml 

Blade  MS 

IM  hS2iZ*i*a  rf  pj*  ahhhi 

T.ZZZUi  1 

9i*jzbj sif 

mm-iuv*  i 

mm.mjm  i 

-i?l  1 

l*P-  = 

HTlItl 

93.1*1  j$  91  1 

sztjiutr 

lf.il  PJ9S  1 

W.?19Jt?  1 

aril  i 

H91 

iUdiff l 

icsahfl 

mn.w  i 

m.mjsa 

mpM*  i 

i 

sari  i 

N9I 

m&m 

Mil  SC 

mjmjm  i 

atnuia 

mslUW  1 

m.mjm  i 

1311  1 

s  »  119 

~  Cad.  CU.  Parian 

StbaMa.  1.3 

ffl&W  1 

SU9EZ.ZN 

ia*  i 

nM&m  i 

i 

i*§i  i 

LLTtL-l 

-o  s 

1 

syiw  i 

m.m.m  i 

ei«  i 

1911 

1  ITTI'l 

PM  I  Iasi* .  :  SC 

1 

muss  i 

1 

1991 

rotoh.-l 

msirnaw  i 

3240,101  JW 

3103401,324 

3310420401 

35% 

201% 

Referring  to  Table  6,  adding  the  technologies  also  improves  overall  maintenance 
implementation  process  ROI,  as  indicated  by  the  positive  difference  between  the  overall 
To-Be  ROI  (201%)  and  the  overall  As-ls  ROI  (35%)  numbers  in  the  lower  right  corner  of 
Table  6.  Adding  these  technologies  also  improves  each  of  the  individual  processes,  as 
indicated  by  the  increases  in  the  To-Be  ROI  values  over  the  As-IS  ROI  values  in  Table 
5.  The  range  of  improvements  across  individual  processes  is  large,  varying  from  6%  to 
466%  (Final  Install,  Closeout  SC),  although  not  as  wide  as  in  the  preparation  for 
maintenance  processes. 


Although  the  same  KVA  modeling  process  was  applied  to  ship  maintenance  in 
both  the  U.S.  and  the  Royal  Dutch  navies,  the  KVA  models  have  important  differences 
that  complicate  the  comparison  of  their  results.  For  example,  the  process  steps  are 
different  and  the  amount  of  field  data  available  to  calibrate  the  models  differ 
significantly.  Therefore,  any  comparisons  can  only  be  preliminary  at  this  point.  However, 
comparison  can  reveal  some  apparent  similarities  and  differences  between  the 
scenarios  that  are  of  interest.  Table  7  shows  the  overall  baseline  (existing  processes) 
and  technology-improved  ROI  for  the  two  U.S.  Navy  scenarios  and  the  Royal  Dutch 
Navy  scenario. 
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Table  7.  Return  on  Investment:  Baseline  and  Technology 

Adoption  Scenarios 


Baseline 
Overall  ROI 

Technology- 
adopted 
Overall  ROI 

US  Navy-SHIPMAIN 

(preparation  for 
maintenance  phases) 

-27% 

2019% 

US  Navy-SHIPMAIN 

(implementation  phases) 

35% 

201% 

Royal  Dutch  Navy 

(Damen  experience 
extrapolation) 

35% 

274% 

The  three  scenarios  have  some  similarities.  For  all  three,  overall  returns  on 
investment  after  technology  adoption  are  positive  and  large.  This  supports  the  adoption 
of  advanced  technologies,  such  as  3D  laser  scanning  technology,  3D  PDF  models,  and 
collaborative  product  lifecycle  management,  to  improve  the  efficiency  of  resource  use. 
The  scenarios  also  have  potentially  significant  differences.  The  technology-adoption 
scenario  for  the  preparation  for  maintenance  phases  of  the  U.S.  scenario  has  a  much 
higher  overall  ROI  than  the  ROIs  for  the  maintenance  implementation  phases  of  the 
U.S.  or  the  Dutch  scenario  (2,019%  »  201%  or  274%).  Several  factors  could  explain 
these  differences. 

•  The  preparation  for  maintenance  phases  of  the  U.S.  scenario  have 
significantly  lower  ROI  in  the  As-ls  (without  technology)  condition  (-27%  > 
35%).  This  suggests  that  inefficiencies  in  the  preparation  for  maintenance 
processes  provided  more  and  larger  opportunities  for  improvement. 

•  The  individual  preparation  for  maintenance  processes  that  increased  the 
most  (see  Table  5)  such  as  Generate  Drawings  and  Conduct  Shipcheck 
are  very  labor  intensive  and,  therefore,  costly,  providing  large 
opportunities  for  cost  reduction  through  technology  adoption. 

•  Several  of  the  individual  maintenance  implementation  processes  (Table  6) 
are  labor  intensive,  but  less  impacted  by  technology  (e.g.,  Install 
Shipcheck),  thereby  making  those  changes  in  ROI  less  dramatic. 
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•  The  preparation  for  maintenance  phases  of  the  U.S.  scenario  could  be 
more  optimistic  in  their  projections  than  the  other  scenarios. 

•  The  estimates  of  process  changes  may  use  different  assumptions. 

•  Technologies  adopted  in  the  preparation  for  maintenance  phases  of  the 
U.S,  scenario  may  make  much  larger  improvements  in  processes  than 
those  in  the  maintenance  implementation  phases  of  the  U.S.  or  the  Dutch 
scenario. 

•  The  Dutch  case  does  not  use  all  of  the  capabilities  of  the  CPLM,  thereby 
making  it  more  incremental  than  the  U.S,  scenarios,  where  all  the 
capabilities  of  the  CPLM  were  projected  to  be  used.  Also,  3D  PDF  has 
more  limited  capabilities  for  integration  with  the  CPLM  logistics  package 
when  compared  to  the  integration  of  3D  LST  capabilities  for  broader 
usage  in  requirements  analysis,  planning  for  maintenance,  and  tracking  of 
parts  in  the  supply  chain  and  across  suppliers  and  contractors.  This  can 
partially  explain  the  lower  ROI  for  the  Dutch  technology-adopted  scenario 
than  the  U.S.  preparation  for  maintenance  scenario. 

•  The  projections  of  the  impacts  on  the  maintenance  implementation  phases 
of  the  U.S,  scenario  and  the  Dutch  scenario  may  be  rather  conservative 
based  on  research  into  the  actual  successful  implementation  of  other 
modern  technologies,  such  as  RFID  in  inventory  management.  In  a  study 
of  the  actual  use  of  passive  RFID  in  two  military  warehouses  in  the  Korean 
Air  Force  and  Army,  the  actual  ROIs  from  use  of  the  RFID  technology 
were  more  than  triple  the  projected  impact  of  the  use  of  the  technology  in 

a  separate  study  of  the  U.S.  Navy  (Courtney,  1997).  The  Korean  ROIs 
after  actual  implementation  of  the  RFID  technology  ranged  from  610%  to 
576%,  compared  to  the  projected  returns  anticipated  from  the 
implementation  of  the  same  technology  in  the  U.S.  Navy,  which  ranged  up 
to  133%.  The  implication  is  that  actual  successful  implementation  of 
information  technology  in  a  military  may  exceed  projections  of  the 
potential  impacts  of  the  technology.  It  follows  that  the  current  research  on 
the  impacts  of  CPLM  and  3D  LST  or  3D  PDF  may  be  more  conservative 
than  the  reality  once  these  technologies  are  actually  implemented  on  a 
wide-scale  basis. 

I.  Integrated  Risk  Management  Modeling  and  Results 

Through  the  use  of  Monte  Carlo  simulation,  the  resulting  stochastic  KVA  ROK 
model  yielded  a  distribution  of  values  rather  than  a  point  solution.  Thus,  simulation 
models  analyze  and  quantify  the  various  risks  and  uncertainties  of  each  program.  The 
result  is  a  distribution  of  the  ROKs  and  a  representation  of  the  project’s  volatility. 
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In  real  options,  the  analyst  assumes  that  the  underlying  variable  is  the  future 
benefit  minus  the  cost  of  the  project.  An  implied  volatility  can  be  calculated  through  the 
results  of  a  Monte  Carlo  simulation.  The  results  for  the  IRM  analysis  will  be  built  on  the 
quantitative  estimates  provided  by  the  KVA  analysis.  The  IRM  will  provide  defensible 
quantitative  risk  analytics  and  portfolio  optimization  suggesting  the  best  way  to  allocate 
limited  resources  to  ensure  the  highest  possible  value  over  time. 

The  first  step  in  real  options  is  to  generate  a  strategic  map  through  the  process  of 
framing  the  problem.  Based  on  the  overall  problem  identification  occurring  during  the 
initial  qualitative  management  screening  process,  certain  strategic  options  would 
become  apparent  for  each  particular  project.  The  strategic  options  could  include,  among 
other  things,  the  option  to  wait,  expand,  contract,  abandon,  switch,  stage-gate,  and 
choose. 

Risk  analysis  and  real  options  analysis  assume  that  the  future  is  uncertain  and 
that  decision-makers  have  the  ability  to  make  midcourse  corrections  when  these 
uncertainties  become  resolved  or  risk  distributions  become  known.  The  analysis  is 
usually  done  ahead  of  time  and,  thus,  ahead  of  such  uncertainty  and  risks.  Therefore, 
when  these  risks  become  known,  the  analysis  should  be  revisited  to  incorporate  the 
information  in  decision-making  or  to  revise  any  input  assumptions.  Sometimes,  for  long- 
horizon  projects,  several  iterations  of  the  real  options  analysis  should  be  performed, 
where  future  iterations  are  updated  with  the  latest  data  and  assumptions. 

Understanding  the  steps  required  to  undertake  an  integrated  risk  analysis  is  important 
because  it  provides  insight  not  only  into  the  methodology  itself  but  also  into  how  it 
evolves  from  traditional  analyses,  showing  where  the  traditional  approach  ends  and 
where  the  new  analytics  start. 

The  risk  simulation  step  required  in  the  IRM  provides  us  with  the  probability 
distributions  and  confidence  intervals  of  the  KVA  methodology’s  resulting  ROI  and  ROK 
results.  Further,  one  of  the  outputs  from  this  risk  simulation  is  volatility,  a  measure  of 
risk  and  uncertainty,  which  is  a  required  input  into  the  real  options  valuation 


ACQUISITION  RESEARCH 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY 
NAVAL  POSTGRADUATE  SCHOOLA 


-36- 


computations.  In  order  to  assign  input  probabilistic  parameters  and  distributions  into  the 
simulation  models,  we  relied  on  the  U.S.  Air  Force’s  Cost  Analysis  Agency  (AFCAA) 
handbook  as  seen  in  Figure  4. 11  In  the  handbook,  the  three  main  distributions 
recommended  are  the  triangular,  normal,  and  uniform  distributions.  We  choose  the 
triangular  distribution  because  the  limits  (minimum  and  maximum)  are  known,  and  the 
shape  of  the  triangular  resembles  the  normal  distribution,  with  the  most  likely  values 
having  the  highest  probability  of  occurrence  and  the  extreme  ends  (minimum  and 
maximum  values)  having  considerably  lower  probabilities  of  occurrence.  Also,  the 
triangular  distribution  was  chosen  instead  of  the  normal  distribution  because  the  latter’s 
tail  ends  extend  toward  positive  and  negative  infinities,  making  it  less  applicable  in  the 
model  we  are  developing.  Finally,  the  AFCAA  also  provides  options  for  left  skew,  right 
skew,  and  symmetrical  distributions.  In  our  analysis,  we  do  not  have  sufficient  historical 
or  comparable  data  to  make  the  proper  assessment  of  skew  and,  hence,  revert  to  the 
default  of  a  symmetrical  triangular  distribution.  Using  these  AFCAA  guidelines,  which 
are  presented  as  15%,  Mean,  and  85%  values  (Figure  6),  we  imputed  the 
corresponding  minimum  (min),  most  likely  (likely),  and  maximum  (max)  values  required 
in  setting  up  the  Triangular  distributions  (Figure  7).12 


11  Available  at  http://www.afhra.af.mil/factsheets/factsheet.asp7icM4009 

12  Using  the  Triangular  distribution’s  probability  density  function  (PDF),  we  simply  compute  the  cumulative 
distribution  function  (CDF).  In  mathematics  and  Monte  Carlo  simulation,  a  PDF  represents  a  continuous 
probability  distribution  in  terms  of  integrals.  If  a  probability  distribution  has  a  density  of  f(x),  then  intuitively 
the  infinitesimal  interval  of  [x,  x  +  dx]  has  a  probability  of  f(x)  dx.  The  PDF,  therefore,  can  be  seen  as  a 
smoothed  version  of  a  probability  histogram;  that  is,  by  providing  an  empirically  large  sample  of  a 
continuous  random  variable  repeatedly,  the  histogram,  using  very  narrow  ranges,  will  resemble  the 
random  variable’s  PDF.  The  probability  of  the  interval  between  [a,  b]  is  given  by  fix)  dx,  which  means 
that  the  total  integral  of  the  function  f  must  be  1 .0.  The  CDF  is  denoted  as  F(x)  =  P(X  <  x),  indicating  the 
probability  of  X  taking  on  a  less  than  or  equal  value  to  x.  Every  CDF  is  monotonically  increasing,  is 
continuous  from  the  right,  and  at  the  limits,  and  has  the  following  properties:  Hmx ,  r(x)  =  o  and 

iimx^+a,  f(x)  =  i.  Further,  the  CDF  is  related  to  the  PDF  by  Fib)  -  Fia)  =  p(a<x  <b)  =  fbafix)dx,  where  the 
PDF  function  f  is  the  derivative  of  the  CDF  function  F.  Using  these  relationships,  we  can  impute  the  min, 
likely,  and  max  values  from  the  mean,  and  the  15th  and  85th  percentiles  that  were  provided  by  the 
AFCAA. 
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AFCAA  Cost  Risk  Analysis  Handbook 
Table  2-5  Default  Bounds  for  Subjective  Distributions 


Distribution. 

Point 

Estimate 

Interpreta 

tioi 

Point 

Estimate  and 
Probability 

Mean 

15% 

85% 

Triangle  [,ow  T.eA 

Mode 

1.0  (75V'*) 
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1.041 
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1.000 

0.Z2A 
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T liangle  Low  Right 
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1.0(25%) 
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0.959 

1.305 

Tiian&Le  Med  Left 

Mode 

1.0(75%) 

0.796 
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1.069 

Triangle  Med 

Mode 

1.0  (50%) 

1  000 

0.723 
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1  204 
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1508 
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0347 
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1.0  (50%) 
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Triangle  EHigh.  LefV 
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0300 
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Mode 

1.0  (50%) 

1  004 

0509 

1-500 
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1.0(25%) 

1307 
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IS>14 

Figure  6.  U.S.  Air  Force  Cost  Analysis  Agency  (US.  AFCAA)  Handbook’s  Probability 

Risk  Distribution  Spreads 

(available  at  http://www.afhra.af. mil/factsheets/factsheet.asp?id=1 4009) 


Symmetrical  Spreads  (Tringular  Distribution) 


Min 

Likely 

Max 

1  High 

0.1421 

1  0000 

1.8579 

2  Above  Average 

0.2648 

1  0000 

1.7352 

3  Medium 

0.3875 

1.0000 

1.6125 

4  Average 

0.5103 

1.0000 

1.4898 

5  Low 

0.6330 

1.0000 

1.3670 

6  Very  Low 

0.7558 

1.0000 

1.2443 

7  Completed/NC 

1.0000 

1  0000 

1.0000 

1  ♦/-  3  Execution 

-3 

3 

2  ♦/-  2  Execution 

-2 

2 

3  ♦/- 1  Execution 

-1 

1 

4  Completed/NC 

0 

0 

Figure  7.  U.S.  AFCAA  Handbook’s  Probability  Risk  Distribution  Spreads 

It  is  important  to  understand  why  it  is  necessary  to  apply  uncertainty  to  the 
model.  Because  the  KVA  process  provided  a  point  value  for  each  quantity,  even  though 
there  is  some  uncertainty  in  the  estimates  provided  by  the  SMEs,  application  of  the 
appropriate  statistical  distributions  of  input  is  used  to  restore  the  real  world’s  uncertainty 
to  the  model.  Having  inputs  from  only  three  experts,  as  opposed  to  hundreds  of 
estimates,  and  rather  than  using  these  three  discrete  inputs,  we  applied  the  lessons 
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learned  in  cost  estimating  as  reflected  in  the  Air  Force  handbook  as  a  good  starting 
point  for  representing  the  uncertainty  and  reflecting  it  in  the  simulations. 

Next,  using  the  developed  KVA  model,  risk  simulation  probabilistic  distributional 
input  parameters  are  inserted  into  the  three  main  variables:  Percentage  Automation, 
Time  Process  Is  Executed,  and  Average  Time  to  Complete  (Figure  8). 13  A  risk 
simulation  of  10,000-1,000,000  simulation  trials  was  run  to  obtain  the  results.14 

Two  sets  of  results  important  in  the  simulation  analysis  are  volatility  and 
probability  confidence  intervals.  The  simulation  statistics  obtained  after  running  a 
simulation  can  be  seen  in  Figure  9,  where  the  main  variable  of  interest  is  the  coefficient 
of  variation,  which  in  this  case  is  used  as  a  proxy  for  volatility.15  The  average  volatilities 
are  between  54%  and  87%.  To  put  this  into  perspective,  the  annualized  volatility  of  blue 
chip  stocks  (e.g.,  IBM  or  Microsoft)  is  typically  between  15%  and  30%,  whereas  higher 
risk  companies  (stocks  with  low  market-to-book  ratios,  low  price-to-earnings  ratios,  or 
startups)  have  their  stocks’  volatilities  above  50%,  and  highly  speculative  derivatives 
may  have  volatilities  upwards  of  100%. 

The  probability  confidence  intervals  will  be  used  and  discussed  in  a  later  section 
within  the  realm  of  real  options  valuation. 

At  this  point  in  the  analysis,  a  proxy  for  revenues  and  volatility  has  been 
identified,  as  well  as  the  numerators  and  denominators  for  the  ship  maintenance 


13  The  Monte  Carlo  Risk  Simulation  was  performed  using  Risk  Simulator  (version  2012)  software  by  Real 
Options  Valuation,  Inc.  (www.realoptionsvaluation.com),  and  screenshots  provided  are  with  permission 
from  the  software  developers. 

14  Different  numbers  of  trials  were  run  to  calibrate  the  precision  of  the  model  and  to  check  for  model 
convergence. 

15  The  coefficient  of  variation  is  simply  defined  as  the  ratio  of  standard  deviation  to  the  mean,  where  risks 
are  common  size.  As  standard  deviation  is  the  measure  of  the  spread  or  dispersion  of  the  data  around  its 
mean,  it  is  oftentimes  used  as  a  measure  of  uncertainty  and,  when  divided  by  the  average  of  the 
distribution,  it  becomes  a  relative  measure  of  risk,  without  any  units.  This  measure  of  risk  or  dispersion  is 
applicable  when  the  variables’  estimates,  measures,  magnitudes,  or  units  differ,  and  can  be  used  as  a 
proxy  for  volatility  of  the  project. 
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program.  The  next  step  is  to  define  or  frame  the  alternatives  and  approaches  to 
implementing  3D  PDF  and  Logistics  Team  Centers,  namely,  strategic  real  options.  The 
questions  that  can  be  answered  include  the  following:  What  are  the  options  involved?; 
How  should  these  new  processes  be  best  implemented?:  Which  decision  pathway  is 
optimal?;  and  How  much  is  the  program  worth  to  the  DoD? 
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Figure  8.  Risk  Simulation  Probability  Distribution  Parameters 
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Figure  9.  Risk  Simulated  Volatility 
J.  IRM:  Why  Strategic  Real  Options? 

As  described  previously,  an  important  step  in  performing  IRM  is  the  application  of 
Monte  Carlo  risk  simulation.  By  applying  Monte  Carlo  risk  simulation  to  simultaneously 
change  all  critical  inputs  in  a  correlated  manner  within  a  model,  researchers  can 
identify,  quantify,  and  analyze  risk.  The  question  then  is,  what  next?  Simply  quantifying 
risk  is  useless  unless  it  can  be  managed,  reduced,  controlled,  hedged,  or  mitigated. 
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This  is  where  strategic  real  options  analysis  comes  in.  Think  of  real  options  as  a 
strategic  road  map  for  making  decisions. 

The  real  options  approach  incorporates  a  learning  model,  such  that  the  decision¬ 
maker  makes  better  and  more  informed  strategic  decisions  when  some  levels  of 
uncertainty  are  resolved  through  the  passage  of  time,  actions,  and  events.  The 
combination  of  the  KVA  methodology,  to  monitor  the  performance  of  given  options,  and 
the  adjustments  to  real  options  as  leaders  learn  more  from  the  execution  of  given 
options,  provides  an  integrated  methodology  to  help  military  leaders  hedge  their  bets 
while  taking  advantage  of  new  opportunities  over  time.  Traditional  analysis  assumes  a 
static  investment  decision,  and  assumes  that  strategic  decisions  are  made  initially  with 
no  recourse  to  choose  other  pathways  or  options  in  the  future.  Real  options  analysis 
can  be  used  to  frame  strategies  to  mitigate  risk,  to  value  and  find  the  optimal  strategy 
pathway  to  pursue,  and  to  generate  options  to  enhance  the  value  of  the  project  while 
managing  risks.  Imagine  real  options  as  a  guide  for  navigating  through  unfamiliar 
territory,  providing  road  signs  at  every  turn  to  direct  drivers  in  making  the  best  and  most 
informed  driving  decisions.  This  is  the  essence  of  real  options.  From  the  options  that  are 
framed,  Monte  Carlo  simulation  and  stochastic  forecasting,  coupled  with  traditional 
techniques,  are  applied.  Then,  real  options  analytics  are  applied  to  solve  and  value 
each  strategic  pathway  and  an  informed  decision  can  be  made.16  See  the  Appendix  B 
for  more  details  on  IRM  and  real  options  analysis. 

K.  IRM:  Framing  the  Real  Options 

As  part  of  the  first  round  of  preliminary  analysis,  Figure  10  illustrates  some  of  the 
potential  implementation  paths  for  3D  PDF/Logistics  TC.  Clearly  some  of  the  pathways 
and  flexibility  strategies  may  be  refined  and  updated  through  the  passage  of  time, 


16  The  pathways  can  be  valued  using  partial  differential  closed-form  equations,  lattices,  and  simulation. 
The  Real  Options  SLS  software,  version  2012  (B),  by  Real  Options  Valuation,  Inc. 
(www.realoptionsvaluation.com),  is  used  to  value  these  options  with  great  ease.  Monte  Carlo  risk 
simulations  were  performed  using  the  Risk  Simulator  software,  version  2012  (B),  created  by  the  same 
organization. 
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actions,  and  events.  With  the  evolution  of  the  implementation,  valuable  information  is 
obtained  to  help  in  further  fine-tuning  the  implementation  and  decision  paths. 

For  the  preliminary  analysis,  the  following  options  were  identified,  subject  to 
modification: 

Option  A:  As-ls  Base  Case.  The  ROI  for  this  strategic  path  is  computed  using  the 
baseline  KVA  and  this  represents  the  current  Royal  Dutch  Navy  ship 
maintenance  process  (i.e.,  no  newly  added  technologies). 

Option  B:  Execute  and  implement  3D  PDF  and  Logistics  package  immediately 
across  all  Royal  Dutch  Navy  ship  maintenance  processes.  That  is,  take  the  risk 
and  execute  on  a  larger  scale,  where  you  would  spend  the  initial  investments  and 
continuing  maintenance  expenses  required  and  take  on  the  risks  of  any  potential 
failure,  but  reap  the  rewards  of  the  new  processes’  savings  quickly  and 
immediately.  The  analysis  is  represented  as  the  current  RDN  process  altered  to 
reflect  what  we  estimate  to  be  the  impacts  of  adopting  both  a  Logistics  package 
and  3D  PDF  models. 

Option  C:  This  represents  the  current  RDN  process  altered  to  reflect  what  we 
estimate  to  be  the  impacts  of  adopting  3D  PDF  models  and  managing  them  in  a 
Team  Center  or  similar  product.  This  technology  was  chosen  largely  because 
Damen  is  developing  and  pursuing  the  use  of  this  technology. 

Option  D:  This  implementation  pathway  represents  the  current  RDN  process 
altered  to  reflect  what  we  estimate  to  be  the  impacts  of  managing  using  a 
Logistics  module  in  a  Team  Center  or  similar  product.  This  technology  was 
chosen  partially  because  it  was  a  technology  that  Damen  considered,  but  chose 
not  to  purchase. 

Option  E:  Proof  of  Concept  approach,  that  is,  to  execute  large-scale 
implementation  of  3D  PDF  and  Logistics  Module  in  TC  only  after  an  initial  Proof 
of  Concept  (POC)  shows  promising  results.  If  POC  turns  out  to  be  a  failure,  we 
walk  away  and  exit  the  program,  and  losses  are  minimized  and  limited  to  the 
initial  POC  expenses.  Proceed  to  full  implementation  in  POC  programs  first  and 
then  expand  in  sequential  fashion  to  other  programs,  based  on  where  best  ROI 
estimates  are  shown. 

Option  F:  Proof  of  Concept  on  3D  PDF  only.  Assuming  the  POC  works  and  3D 
PDF  is  executed  within  a  few  programs  successfully,  the  learning  and  experience 
obtained  becomes  valuable  and  allows  the  shipyard  to  expand  its  use  into  many 
other  programs  or  perhaps  across  the  Royal  Dutch  Navy. 
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Option  G:  Proof  of  Concept  on  Logistics  Module  in  TC  only.  Assuming  the  POC 
works  and  Logistics  Module  is  executed  within  a  few  programs  successfully,  the 
learning  and  experience  obtained  becomes  valuable  and  allows  the  shipyard  to 
expand  its  use  into  many  other  programs  or  perhaps  across  the  Royal  Dutch 
Navy. 

Figure  1 1  shows  the  preliminary  input  assumptions  and  Figure  12  shows  the 
computed  return  on  investment  results  and  strategic  real  options  results.  For  instance, 
the  following  inputs  were  assumed: 

PV  Asset.  This  is  the  net  total  benefits  or  proxy  revenues  (numerator)  obtained 
from  the  KVA  analysis  under  each  of  the  various  options  as  outlined  previously. 

Implementation  Cost.  This  is  the  total  cost  to  implement  each  of  the  options  (e.g., 
3D  PDF  only,  3D  PDF  with  Logistics  Module  TC,  or  Logistics  Module  TC  only). 

Maturity.  This  is  the  time  to  perform  the  proof  of  concept  stage,  denoted  in  years. 

Risk-Free  Rate.  This  is  the  annualized  U.S.  Treasury  rate  used  as  a  proxy  of  a 
risk-free  asset.  This  rate  is  used  to  discount  the  future  cash  flows  in  the  risk- 
neutral  options  model.  We  use  a  risk-free  rate  as  the  risk  has  already  been 
accounted  for  in  the  risk  simulation  and  volatility  estimates.  Figure  13  illustrates 
the  U.S.  Treasury  security  interest  rates  used  as  a  proxy  for  the  risk-free  rate 
used  in  the  analysis. 

Volatility.  This  is  the  annualized  volatility  estimate  obtained  from  Monte  Carlo  risk 
simulation  in  the  previous  step  by  using  the  AFCAA  risk  spreads  as  a  proxy. 

Dividend  Rate.  This  variable  is  typically  not  used  but  is  available  should  the  need 
arise.  Briefly,  it  measures  the  annualized  percentage  rate  of  the  opportunity  cost 
of  investing  at  a  future  time  instead  of  immediately. 
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IRM:  Strategic  Flexibility  Real  Options 
Results 


Figure  12  shows  the  results  of  the  strategic  real  options  flexibility  values  and 
compares  them  against  the  KVA  ROI  values.  Options  B  ($154.1  million  at  278%  ROI) 
and  E  ($156.5  million  at  282%  ROI)  of  implementing  both  3D  PDF  and  Logistics  Module 
TC  return  the  highest  ROI  and  total  strategic  value,  and  both  provide  a  significant  value- 
add  above  and  beyond  Option  A’s  As-ls  condition  ($31 .9  million  at  35%  ROI).  As 
Options  B  and  E  are  the  most  significant,  stage-gating  the  implementation  over  several 
phases  yields  a  slightly  higher  value  (Option  E  exceeds  Option  B  by  about  $2.4  million). 

In  addition,  Figure  13  shows  the  Monte  Carlo  risk  simulation  results  on  the  real 
options  values.  For  instance,  in  comparing  Options  E  and  F,  there  is  a  94%  probability 
that  Option  E,  which  has  a  sequentially  phased  implementation  of  both  3D  PDF  and 
Logistics  Module  TC,  provides  a  better  return  than  Option  F.  In  comparing  Option  E  with 
Option  B,  there  is  a  95%  confidence  that,  even  with  all  the  uncertainties  in  the  collected 
data  and  risks  of  implementation  success,  including  uncertainties  of  whether  the 
estimated  returns  will  materialize  and  so  forth,  there  is  at  least  a  $1.27  million  net 
advantage  in  going  with  Option  E.  Therefore,  it  is  better  to  sequentially  phase  and 
stage-gate  the  implementation  over  several  years,  allow  the  ability  to  exit  and  abandon 
further  stages  if  events  unfold  and  uncertainties  become  resolved,  so  that  further 
investment  in  the  technology  no  longer  makes  sense. 

As  additional  information,  with  KVA  baseline  of  Option  A,  we  see  that  without 
doing  any  implementations,  there  is  still  a  4.7%  probability  that  staying  As-ls  returns 
negative  ROIs,  and  even  in  the  best  case  analysis  there  is  less  than  a  5%  probability 
that  ROI  for  the  base  case  will  ever  exceed  93%. 

The  final  two  charts  in  Figure  14  show  that  the  risk  simulated  real  options  value 
has  an  expected  value  (mean)  of  $195  million  with  a  corresponding  average  ROI  of 
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363%.  Finally,  Figure  15  shows  the  comprehensive  simulated  risk  statistics  of  the 
various  option  scenarios. 


Figure  10.  Strategic  Real  Options  Implementation  Pathways  and  Options 
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ANALYSIS  RESULTS 
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Figure  15.  Risk  Simulation  Statistics  and  Percentiles 
A.  Summary  Results  of  the  IRM  Analysis 

Integrated  risk  management  and  strategic  real  options  methodologies  were 
applied  to  the  KVA-SD  results  and  the  results  indicate  that  Option  B  had  a  value  of 
$154.1  million  (278%  ROI)  and  Option  E  had  a  value  of  $156.5  million  (282%  ROI) 
where  both  options  indicate  that  implementing  3D  PDF  and  Logistics  Module  TC  return 
the  highest  ROI  and  total  strategic  value,  and  both  provide  a  significant  value-add 
above  and  beyond  Option  A’s  As-ls  condition  with  a  value  of  $31 .9  million  (35%  ROI). 

As  Options  B  and  E  are  most  significant,  we  know  that  implementation  of  3D  PDF  and 
Logistics  Module  TC  return  the  highest  value,  and  when  implemented  over  time  in  a 
stage-gate  process  over  several  phases,  would  yield  a  slightly  higher  value  (Option  E 
exceeds  Option  B  by  about  $2.4  million).  Therefore,  we  conclude  that  3D  PDF  and 
Logistics  Module  TC  implemented  in  a  phased  stage-gate  environment  would  yield  the 
best  results.  In  comparing  Option  E  with  Option  B,  there  is  a  95%  probability,  even  with 
all  the  uncertainties  in  the  collected  data  and  risks  of  implementation  success,  as  well 
as  uncertainties  of  whether  the  estimated  returns  will  materialize,  there  is  a  $1.27  million 
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net  advantage  in  going  with  Option  E  to  sequentially  phase  and  stage-gate  the 
implementation  over  several  years,  and  allow  the  ability  to  exit  and  abandon  further 
stages  if  events  unfold  and  uncertainties  become  resolved,  so  that  further  investment 
the  technology  no  longer  makes  sense. 
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IV. 


Conclusions 


We  collected  new  data  on  ship  maintenance  processes  and  the  use  and  adoption 
of  technologies  in  ship  maintenance  by  the  Royal  Dutch  Navy  and  Damen  Shipbuilding. 
The  data  were  used  to  build  and  calibrate  a  system  dynamics  model  of  Royal  Dutch 
Naval  ship  maintenance.  Model  simulations  of  four  technology  adoption  scenarios, 
reflecting  the  use  of  two  available  or  developing  technologies,  generated  estimates  of 
maintenance  operations  behavior  that  were  imported  into  knowledge  value  added 
models.  The  four  technology  adoption  scenarios  were  then  modeled  in  the  KVA  models. 
The  KVA  models  estimated  the  returns  on  investment  for  individual  processes  and  ship 
maintenance  as  a  whole  for  each  scenario.  Results  were  analyzed  to  reveal  the  relative 
improvement  provided  by  individual,  and  combinations  of,  technologies. 

The  results  of  this  study,  in  combination  with  prior  studies,  make  it  evident  that 
the  technologies  under  review  will  make  large  contributions  to  cost  reductions  in  ship 
maintenance  processes.  These  conclusions  are  supported  by  the  comparative  analysis 
of  the  Dutch  experience  with  similar  supporting  technologies.  There  appears  to  be  no 
empirical  evidence  that  would  serve  as  an  impediment  to  adopting  the  technologies  in 
the  near  term  rather  than  the  longer  term.  We  recommend  an  immediate  adoption  of  the 
3D  LST  and  CPLM  technologies  to  support  ship  maintenance  processes. 

A.  Implications  for  Acquisition  Practice 

The  current  research  has  significant  implications  for  acquisition  practice.  First, 
the  conclusions  support  multiple  previous  investigations  that  recommend  the  adoption 
of  available  information  technologies  to  reduce  the  costs  of  U.S.  Navy  ship 
maintenance.  Second,  multiple  significantly  different  technologies  (e.g.,  3D  LST,  3D 
PDF,  logistics  support)  can  improve  ship  maintenance  operations.  Third,  among  those 
studied,  the  expensive  information  technologies  were  found  to  benefit  high-cost 
processes  the  most:  for  example,  where  labor  can  be  replaced  with  technology.  Doing 
so  reduces  costs  and  increases  production  rates  by  reducing  cycle  times.  This  implies 
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that,  if  technology  adoption  efforts  are  to  be  prioritized,  those  with  labor-intensive 
processes  that  can  be  replaced  with  technology  should  be  given  higher  priority.  The  real 
options  analysis  of  implementation  strategies  demonstrated  that  some  technologies  (3D 
PDF  in  this  case)  can  dominate  the  value  space  and  that  phased  implementation  adds 
value  compared  to  one-step  implementation.  The  results  of  the  current  work 
recommend  a  careful  investigation  of  available  technologies  and  how  they  improve 
operations,  followed  by  a  phased  development  and  implementation  of  the  adoption  of 
the  chosen  technologies. 

B.  Implications  for  Research 

The  results  of  the  three  KVA-based  studies  varied  significantly.  A  likely  cause  is 
the  difficulty  in  accurately  forecasting,  in  quantitative  terms,  the  impacts  of  new 
technologies  on  specific  processes.  The  use  of  data  and  information  from  organizations 
that  are  actively  developing  and  adopting  information  technologies  (Damen)  and 
performing  operations  similar  to  those  performed  by  the  U.S.  Navy  (Royal  Dutch  Navy) 
proved  to  be  very  valuable  in  improving  the  models  (e.g.,  by  adding  the  3D  PDF 
technology).  Therefore,  further  refinement  of  the  models  should  include  actual 
application  data,  such  as  a  study  of  actual  technology  adoptions  by  the  U.S.  Navy. 
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Appendix  A:  Informants  (Alphabetical  Order)  and  Site 

Visit  Locations 


Informants 

Sander  Alles,  Manager  Maintenance  &  Spares,  Damen  Services,  Gorinchem, 

The  Netherlands. 

Hein  van  Ameijden,  Managing  Director,  Damen  Schelde  Naval  Shipbuilding, 
Vlissingen,  The  Netherlands. 

Nico  de  Vries,  KTZT  ir.  Head  of  Corporate  Planning  and  Strategy,  Dutch  Naval 
Maintenance  and  Service  Agency,  The  Netherlands. 

Bert  Geisler,  Business  Development  Director  Shipbuilding,  Siemens  PLM 
Software,  Hamburg,  Germany. 

Paul  J.  Kense,  Advisor  ILS,  Naval  Maintenance  Establishment,  Defense  Materiel 
Organization,  Royal  Dutch  Navy,  Den  Helder,  The  Netherlands. 

Desmond  Kramer,  Manager,  Integrated  Logistic  Support,  Engineering 
Department,  Damen  Schelde  Naval  Shipbuilding,  Vlissingen,  The  Netherlands. 

Randy  Langmead,  Director,  Marine/Federal  Business  Development,  Siemens 
PLM  Software,  Washington,  D.C. 

Niek  Marse,  Integrated  Logistic  System,  Engineering  Department,  Damen 
Schelde  Naval  Shipbuilding,  Vlissingen,  The  Netherlands. 

Michael  Schwind,  Vice  President,  Federal  Sector,  Siemens-UGS  PLM  Software, 
Philadelphia,  PA. 

Ronald  van  Noppen,  KTZE  ir.  Head  of  Material  and  Logistics,  Naval  Maintenance 
and  Service  Agency,  The  Netherlands. 

Frank  Verhelst,  Manager,  Project  Department,  Damen  Schelde  Naval 
Shipbuilding,  Vlissingen,  The  Netherlands. 
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Thijs  Verwoerd,  Project  Manager,  Damen  Services,  Gorinchem,  The 
Netherlands. 

Site  Visit  Locations 

Damen  Schelde  Naval  Shipbuiding  (DSNS) 

Glacisstraat  165 

4381  SE  Vlissingen,  The  Netherlands 

Dutch  Naval  Maintenance  Facilities 
Nieuwe  Haven 

1780  CA  Den  Helder,  The  Netherlands 
Damen  Services 

Industrieterrein  Avelingen  West  20 
4202  MS  Gorinchem,  The  Netherlands 
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Appendix  B.  Primer  on  Risk  Simulation,  Return  on 

Investment,  Strategic  Real  Options,  and 
Portfolio  Optimization — Integrated  Risk 
Management17 

Since  the  beginning  of  recorded  history,  games  of  chance  have  been  a  popular 
pastime.  Even  in  Biblical  accounts,  Roman  soldiers  cast  lots  for  Christ’s  robes.  In  earlier 
times,  chance  was  something  that  occurred  in  nature,  and  humans  were  simply 
subjected  to  it  as  a  ship  is  to  the  capricious  tosses  of  the  waves  in  an  ocean.  Even  up  to 
the  time  of  the  Renaissance,  the  future  was  thought  to  be  simply  a  chance  occurrence 
of  completely  random  events  and  beyond  the  control  of  humans.  However,  with  the 
advent  of  games  of  chance,  human  greed  has  propelled  the  study  of  risk  and  chance  to 
ever  more  closely  mirror  real-life  events.  Although  these  games  were  initially  played 
with  great  enthusiasm,  no  one  actually  sat  down  and  figured  out  the  odds.  Of  course, 
the  individual  who  understood  and  mastered  the  concept  of  chance  was  bound  to  be  in 
a  better  position  to  profit  from  such  games  of  chance.  It  was  not  until  the  mid-1600s  that 
the  concept  of  chance  was  properly  studied,  and  the  first  such  serious  endeavor  can  be 
credited  to  Blaise  Pascal,  one  of  the  fathers  of  the  study  of  choice,  chance,  and 
probability.  Fortunately  for  us,  after  many  centuries  of  mathematical  and  statistical 
innovations  from  pioneers  such  as  Pascal,  Bernoulli,  Bayes,  Gauss,  LaPlace,  and 
Fermat,  and  with  the  advent  of  blazing-fast  computing  technology,  our  modern  world  of 
uncertainty  can  be  explained  with  much  more  elegance  through  methodological, 
rigorous  hands-on  applications  of  risk  and  uncertainty.  Even  as  recent  as  two  and  a  half 
decades  ago,  computing  technology  was  only  in  its  infancy  and  running  complex  and 
advanced  analytical  models  would  have  seemed  a  fantasy;  however,  today,  with  the 
assistance  of  more  powerful  and  enabling  software  packages,  we  have  the  ability  to 
practically  apply  such  techniques  with  great  ease.  For  this  reason,  we  have  chosen  to 


17  This  primer  is  written  by  Dr.  Johnathan  Mun  and  is  based  on  his  two  latest  books,  Modeling  Risk, 
Second  Edition  (Wiley,  2010)  and  Real  Options  Analysis,  Second  Edition  (Wiley,  2006). 
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learn  from  human  history  that  with  innovation  comes  the  requisite  change  in  human 
behavior  to  apply  these  new  methodologies  as  the  new  norm  for  rigorous  risk-benefit 
analysis. 

To  the  people  who  lived  centuries  ago,  risk  was  simply  the  inevitability  of  chance 
occurrence  beyond  the  realm  of  human  control.  Albeit  many  phony  soothsayers  profited 
from  their  ability  to  convincingly  profess  their  clairvoyance  by  simply  stating  the  obvious 
or  reading  the  victims’  body  language  and  telling  them  what  they  wanted  to  hear.  We 
modern-day  humans,  ignoring  for  the  moment  the  occasional  seers  among  us,  with  our 
fancy  technological  achievements,  are  still  susceptible  to  risk  and  uncertainty.  We  may 
be  able  to  predict  the  orbital  paths  of  planets  in  our  solar  system  with  astounding 
accuracy  or  the  escape  velocity  required  to  shoot  a  man  from  the  Earth  to  the  Moon,  or 
drop  a  smart  bomb  within  a  few  feet  of  its  target  thousands  of  miles  away,  but  when  it 
comes  to,  say,  predicting  a  firm’s  revenues  the  following  year,  we  are  at  a  loss.  Humans 
have  been  struggling  with  risk  our  entire  existence,  but  through  trial  and  error,  and 
through  the  evolution  of  human  knowledge  and  thought,  have  devised  ways  to  describe, 
quantify,  hedge,  and  take  advantage  of  risk. 

In  the  U.S.  military  context,  risk  analysis,  real  options  analysis,  and  portfolio 
optimization  techniques  are  enablers  of  a  new  way  of  approaching  the  problems  of 
estimating  return  on  investment  (ROI)  and  the  risk-value  of  various  strategic  real 
options.  There  are  many  new  Department  of  Defense  (DoD)  requirements  for  using 
more  advanced  analytical  techniques.  For  instance,  the  Clinger-Cohen  Act  of  1996 
mandates  the  use  of  portfolio  management  for  all  federal  agencies.  The  Government 
Accounting  Office’s  (1997)  “Assessing  Risks  and  Returns:  A  Guide  for  Evaluating 
Federal  Agencies’  IT  Investment  Decision-Making”  requires  that  IT  investments  apply 
ROI  measures.  DoD  Directive  8115.01  (DoD,  2005)  issued  October  2005  mandates  the 
use  of  performance  metrics  based  on  outputs,  with  ROI  analysis  required  for  all  current 
and  planned  IT  investments.  DoD  Directive  8115.bb  (2006)  implements  policy  and 
assigns  responsibilities  for  the  management  of  DoD  IT  investments  as  portfolios  within 
the  DoD  enterprise  where  they  define  a  portfolio  to  include  outcome  performance 
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measures  and  an  expected  return  on  investment.  The  DoD  Risk  Management  Guidance 
Defense  Acquisition  guidebook  requires  that  alternatives  to  the  traditional  cost 
estimation  need  to  be  considered  because  legacy  cost  models  tend  not  to  adequately 
address  costs  associated  with  information  systems  or  the  risks  associated  with  them. 

In  this  quick  primer,  advanced  quantitative  risk-based  concepts  will  be 
introduced,  namely,  the  hands-on  applications  of  Monte  Carlo  simulation,  real  options 
analysis,  stochastic  forecasting,  portfolio  optimization,  and  knowledge  value  added. 
These  methodologies  rely  on  common  metrics  and  existing  techniques  (e.g.,  return  on 
investment,  discounted  cash  flow,  cost-based  analysis,  and  so  forth),  and  complement 
these  traditional  techniques  by  pushing  the  envelope  of  analytics,  but  do  not  replace 
them  outright.  It  is  not  a  complete  change  of  paradigm,  and  we  are  not  asking  the 
reader  to  throw  out  what  has  been  tried  and  true,  but  to  shift  one’s  paradigm,  to  move 
with  the  times,  and  to  improve  upon  what  has  been  tried  and  true.  These  new 
methodologies  are  used  in  helping  make  the  best  possible  decisions,  allocate  budgets, 
predict  outcomes,  create  portfolios  with  the  highest  strategic  value  and  returns  on 
investment,  and  so  forth,  where  the  conditions  surrounding  these  decisions  are  risky  or 
uncertain.  They  can  be  used  to  identify,  analyze,  quantify,  value,  predict,  hedge, 
mitigate,  optimize,  allocate,  diversify,  and  manage  risk  for  military  options. 

Why  Is  Risk  Important  in  Making  Decisions? 

Before  we  embark  on  the  journey  to  review  these  advanced  techniques,  let  us 
first  consider  why  risk  is  critical  when  making  decisions,  and  how  traditional  analyses 
are  inadequate  in  considering  risk  in  an  objective  way.  Risk  is  an  important  part  of  the 
decision-making  process.  For  instance,  suppose  projects  are  chosen  based  simply  on 
an  evaluation  of  returns  alone  or  cost  alone;  clearly  the  higher  return  or  lower  cost 
project  will  be  chosen  over  lower  return  or  higher  cost  projects. 

As  mentioned,  projects  with  higher  returns  will  in  most  cases  bear  higher  risks. 
And  those  projects  with  immediately  lower  returns  would  be  abandoned.  In  those  cases, 
where  return  estimates  are  wholly  derived  from  cost  data  (with  some  form  of  cost  in  the 
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numerator  and  denominator  of  ROI),  the  best  thing  to  do  is  reduce  all  the  costs,  that  is, 
never  invest  in  new  projects.  The  result  of  this  primary  focus  on  cost  reduction  is  a 
stifling  of  innovation  and  new  ways  of  doing  things.  The  goal  is  not  simply  cost 
reduction.  In  this  case,  the  simplest  approach  is  to  fire  everyone  and  sell  off  all  the 
assets.  The  real  question  that  must  be  answered  is  how  cost  compares  to  desired 
outputs,  that  is,  “cost  compared  to  what”? 

To  encourage  a  focus  on  improving  processes  and  innovative  technologies,  a 
new  way  of  calculating  return  on  investment  that  includes  a  unique  numerator  is 
required.  ROI  is  a  basic  productivity  ratio  that  requires  unique  estimates  of  the 
numerator  (i.e.,  value,  revenue  in  common  units  of  measurement)  and  the  denominator 
(i.e.,  costs,  investments  in  dollars).  ROI  estimates  must  be  placed  within  the  context  of  a 
longer  term  view  that  includes  estimates  of  risk  and  the  ability  of  management  to  adapt 
as  they  observe  the  performance  of  their  investments  over  time.  Therefore,  instead  of 
relying  purely  on  immediate  ROIs  or  costs,  a  project,  strategy,  process  innovation,  or 
new  technology  should  be  evaluated  based  on  its  total  strategic  value,  including  returns, 
costs,  and  strategic  options,  as  well  as  its  risks.  Figures  A.1  and  A.2  illustrate  the  errors 
in  judgment  when  risks  are  ignored.  Figure  A.1  lists  three  mutually  exclusive  projects 
with  their  respective  costs  to  implement,  expected  net  returns  (net  of  the  costs  to 
implement),  and  risk  levels  (all  in  present  values).18  Clearly,  for  the  budget-constrained 
decision-maker,  the  cheaper  the  project,  the  better,  resulting  in  the  selection  of  Project 
X.  The  returns-driven  decision-maker  will  choose  Project  Y  with  the  highest  returns, 
assuming  that  budget  is  not  an  issue.  Project  Z  will  be  chosen  by  the  risk-averse 
decision-maker,  as  it  provides  the  least  amount  of  risk  while  providing  a  positive  net 
return.  The  upshot  is  that,  with  three  different  projects  and  three  different  decision¬ 
makers,  three  different  decisions  will  be  made.  Who  is  correct  and  why? 


18  Risks  can  be  computed  many  ways,  including  volatility,  standard  deviation  of  lognormal  returns,  value 
at  risk,  and  so  forth.  See  Modeling  Risk,  by  Johnathan  Mun  (Wiley,  2010)  for  more  technical  details. 
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Why  is  Risk  Important? 


Name  of  Project 

Cost 

Returns 

Risk 

Project  X 

$50 

$50 

$25 

Project  Y 

$250 

$200 

$200 

Project  Z 

$100 

$100 

$10 

Project  X  for  the  cost  and  budget-constrained  manager 

Project  Y  for  the  returns  driven  and  non  resource-constrained  manager 

Project  Z  for  the  risk-adverse  manager 

Project  Z  for  the  smart  manager 


Figure  A.1.  Why  Is  Risk  Important? 

Figure  A.2  shows  that  Project  Z  should  be  chosen.  For  illustration  purposes, 
suppose  all  three  projects  are  independent  and  mutually  exclusive,  and  that  an 
unlimited  number  of  projects  from  each  category  can  be  chosen,  but  the  budget  is 
constrained  at  $1 ,000.  Therefore,  with  this  $1 ,000  budget,  20  project  Xs  can  be  chosen, 
yielding  $1,000  in  net  returns  and  $500  in  risks,  and  so  forth.  It  is  clear  from  Figure  A.2 
that  Project  Z  is  the  best  project  because  for  the  same  level  of  net  returns  ($1 ,000),  the 
least  amount  of  risk  is  undertaken  ($100).  Another  way  of  viewing  this  selection  is  that 
for  each  $1  of  returns  obtained,  only  $0.1  of  risk  is  involved  on  average,  or  that  for  each 
$1  of  risk,  $10  in  returns  are  obtained  on  average.  This  example  illustrates  the  concept 
of  bang  for  the  buck  or  getting  the  best  value  (benefits  and  costs  both  considered)  with 
the  least  amount  of  risk.  An  even  more  blatant  example  is  if  there  are  several  different 
projects  with  identical  single-point  average  net  benefits  or  costs  of  $10  million  each. 
Without  risk  analysis,  a  decision-maker  should  in  theory  be  indifferent  in  choosing  any 
of  the  projects.  However,  with  risk  analysis,  a  better  decision  can  be  made.  For 
instance,  suppose  the  first  project  has  a  10%  chance  of  exceeding  $10  million,  the 
second  a  15%  chance,  and  the  third  a  55%  chance.  Additional  critical  information  is 
obtained  on  the  riskiness  of  the  project  or  strategy  and  a  better  decision  can  be  made. 
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Adding  an  Element  of  Risk... 

Looking  at  bang  for  the  buck,  X  (2),  Y  (1).  Z  (10).  Project  Z  should 
be  chosen-with  a  SI  .000  budget,  the  following  can  be  obtained: 

Project  X; 
Project  Y: 
Project  Z: 

20  Project  Xs  returning  SI. 000,  with  $500  risk 

4  Project  Xs  returning  S800.  with  $800  risk 

10  Project  Xs  returning  $1,000,  with  $100  risk 

Project  X: 
Project  Y: 
Project  Z: 

For  each  SI  return.  $0.5  risk  is  taken 

For  each  SI  return.  $1 .0  risk  is  taken 

For  each  $1  return.  $0.1  risk  is  taken 

Project  X: 
Project  Y: 
Project  Z: 

For  each  SI  of  risk  taken.  $2  return  is  obtained 

For  each  $1  of  risk  taken.  $1  return  is  obtained 

For  each  $1  of  risk  taken.  $10  return  Is  obtained 

Conclusion: 

Risk  is  important  Foregoing  risks  results  in  making  the  wrong  decision. 

Figure  A.2.  Adding  an  Element  of  Risk 
From  Dealing  With  Risk  the  Traditional  Way  to  Monte  Carlo  Simulation 

Military  and  business  leaders  have  been  dealing  with  risk  since  the  beginning  of 
the  history  of  war  and  commerce.  In  most  cases,  decision-makers  have  looked  at  the 
risks  of  a  particular  project,  acknowledged  their  existence,  and  moved  on.  Little 
quantification  was  performed  in  the  past.  In  fact,  most  decision-makers  look  only  to 
single-point  estimates  of  a  project’s  benefit  or  profitability.  Figure  A.3  shows  an  example 
of  a  single-point  estimate.19  The  estimated  net  revenue  of  $30  is  simply  that,  a  single 
point  whose  probability  of  occurrence  is  close  to  zero.20  Even  in  the  simple  model  shown 
in  Figure  A.3,  the  effects  of  interdependencies  are  ignored,  and  in  traditional  modeling 
jargon,  we  have  the  problem  of  garbage-in,  garbage-out  (GIGO).  As  an  example  of 
interdependencies,  the  units  sold  are  probably  negatively  correlated  to  the  price  of  the 
product,  and  positively  correlated  to  the  average  variable  cost;  ignoring  these  effects  in 
a  single-point  estimate  will  yield  grossly  incorrect  results.  There  are  numerous 


19  We  will  demonstrate  how  KVA,  combined  with  the  traditional  Market  Comparables  valuation  method, 
allows  for  the  monetization  of  benefits  (i.e. ,  revenue). 

20  On  a  continuous  basis,  the  probability  of  occurrence  is  the  area  under  a  curve  (e.g.,  there  is  a  90% 
probability  revenues  will  be  between  $10  million  and  $1 1  million.  However,  the  area  under  a  straight  line 
approaches  zero.  Therefore,  the  probability  of  hitting  exactly  $10.0000  is  close  to  0.00000001%. 
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interdependencies  in  military  options  as  well,  for  example,  the  many  issues  in  logistics 
and  troop  movements  beginning  with  the  manufacturer  all  the  way  to  the  warrior  in  the 
field. 


In  the  commercial  example  below  (Figure  A.3),  if  the  unit  sales  variable  becomes 
1 1  instead  of  10,  the  resulting  revenue  may  not  simply  be  $35.  The  net  revenue  may 
actually  decrease  due  to  an  increase  in  variable  cost  per  unit  while  the  sale  price  may 
actually  be  slightly  lower  to  accommodate  this  increase  in  unit  sales.  Ignoring  these 
interdependencies  will  reduce  the  accuracy  of  the  model. 


Single  Point  Estimate 

Unit  Sales 

Sales  Price 

/  10 

InleiUepeiKJeiijes 

1  rr«ans  GIGO 

Total  Revenue 

/  $100 

Variable  Cost/Unit 

/  $5 

Total  Fixed  Cost 

l  $20 

Total  Cost 

\  $70 

Net  Revenue 

^  $30 

Single  Point  Estimate 

How  confident  are  you  of  the  analysis  outcome? 

This  may  be  dead  wrong! 

Figure  A.3.  Single-Point  Estimates 

One  traditional  approach  used  to  deal  with  risk  and  uncertainty  is  the  application 
of  scenario  analysis.  For  example,  scenario  analysis  is  a  central  part  of  the  capabilities- 
based  planning  approach  in  widespread  use  for  developing  DoD  strategies.  In  the 
commercial  example  above  (Figure  A.3),  suppose  three  scenarios  were  generated:  the 
worst-case,  nominal-case,  and  best-case  scenarios.  When  different  values  are  applied 
to  the  unit  sales,  the  resulting  three  scenarios’  net  revenues  are  obtained.  As  earlier,  the 
problems  of  interdependencies  are  not  addressed  with  these  common  approaches.  The 
net  revenues  obtained  are  simply  too  variable.  Not  much  can  be  determined  from  such 
an  analysis. 
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In  the  military  planning  case,  the  problems  are  exacerbated  by  the  lack  of 
objective  ways  to  estimate  benefits  in  common  units.  Without  the  common-unit  benefits 
analysis,  it  becomes  difficult,  if  not  impossible,  to  compare  the  net  benefits  of  various 
scenarios.  In  addition,  interdependencies  must  be  interpreted  in  a  largely  subjective 
manner,  making  it  impossible  to  apply  powerful  mathematical  and  statistical  tools  that 
enable  more  objective  portfolio  analysis.  The  problem  arises  for  the  top  leaders  in  the 
DoD  to  make  judgment  calls,  selection  among  alternatives  (often  referred  to  as  “trades”) 
about  the  potential  benefits  and  risks  of  numerous  projects  and  technologies 
investments 


A  related  approach  is  to  perform  what-if  or  sensitivity  analysis.  Each  variable  is 
perturbed  a  pre-specified  amount  (e.g.,  unit  sales  is  changed  ±10%,  sales  price  is 
changed  ±5%,  and  so  forth)  and  the  resulting  change  in  net  benefits  is  captured.  This 
approach  is  useful  for  understanding  which  variables  drive  or  impact  the  result  the  most. 
Performing  such  analyses  by  hand  or  with  simple  Excel  spreadsheets  is  tedious  and 
provides  marginal  benefits  at  best.  A  related  approach  that  has  the  same  goals  but 
employs  a  more  powerful  analytic  framework  is  the  use  of  computer-modeled  Monte 
Carlo  simulation  and  tornado  sensitivity  analysis,  where  all  perturbations,  scenarios, 
and  sensitivities  are  run  hundreds  of  thousands  of  times  automatically. 


Therefore,  computer-based  Monte  Carlo  simulation,  one  of  the  advanced 
concepts  introduced  in  this  paper,  can  be  viewed  as  simply  an  extension  of  the 
traditional  approaches  of  sensitivity  and  scenario  testing.  The  critical  success  drivers  or 
the  variables  that  affect  the  bottom-line  variables  the  most,  which  at  the  same  time  are 
uncertain,  are  simulated.  In  simulation,  the  interdependencies  are  accounted  for  by 
using  correlation  analysis.  The  uncertain  variables  are  then  simulated  tens  of  thousands 
of  times  automatically  to  emulate  all  potential  permutations  and  combinations  of 
outcomes.  The  resulting  net  revenues-benefits  from  these  simulated  potential  outcomes 
are  tabulated  and  analyzed.  In  essence,  in  its  most  basic  form,  simulation  is  simply  an 
enhanced  version  of  traditional  approaches  such  as  sensitivity  and  scenario  analysis, 
but  automatically  performed  thousands  of  times  while  accounting  for  all  the  dynamic 
interactions  between  the  simulated  variables.  The  resulting  net  revenues  from 
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simulation,  as  seen  in  Figure  A.4,  show  that  there  is  a  90%  probability  that  the  net 
revenues  will  fall  between  $19.44  and  $41.25,  with  a  5%  worst-case  scenario  of  net 
revenues  falling  below  $19.44.  Rather  than  having  only  three  scenarios,  the  simulation 
created  5,000  scenarios,  or  trials,  where  multiple  variables  are  simulated  and  changing 
simultaneously  (unit  sales,  sale  price,  and  variable  cost  per  unit),  while  their  respective 
relationships  or  correlations  are  maintained. 


Figure  A.4.  Simulation  Results 

Monte  Carlo  simulation,  named  for  the  famous  gambling  capital  of  Monaco,  is  a 
very  potent  methodology.  For  the  practitioner,  simulation  opens  the  door  for  solving 
difficult  and  complex  but  practical  problems  with  great  ease.  Perhaps  the  most  famous 
early  use  of  Monte  Carlo  simulation  was  by  the  Nobel  physicist  Enrico  Fermi 
(sometimes  referred  to  as  the  father  of  the  atomic  bomb)  in  1930,  when  he  used  a 
random  method  to  calculate  the  properties  of  the  newly  discovered  neutron.  Monte 
Carlo  methods  were  central  to  the  simulations  required  for  the  Manhattan  Project, 
where  in  the  1950s  Monte  Carlo  simulation  was  used  at  Los  Alamos  for  early  work 
relating  to  the  development  of  the  hydrogen  bomb  and  became  popularized  in  the  fields 
of  physics  and  operations  research.  The  Rand  Corporation  and  the  U.S.  Air  Force  were 
two  of  the  major  organizations  responsible  for  funding  and  disseminating  information  on 
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Monte  Carlo  methods  during  this  time,  and  today  there  is  a  wide  application  of  Monte 
Carlo  simulation  in  many  different  fields  including  engineering,  physics,  research  and 
development,  business,  and  finance. 

Simplistically,  Monte  Carlo  simulation  creates  artificial  futures  by  generating 
thousands  and  even  hundreds  of  thousands  of  sample  paths  of  outcomes  and  by 
analyzing  their  prevalent  characteristics.  In  practice,  Monte  Carlo  simulation  methods 
are  used  for  risk  analysis,  risk  quantification,  sensitivity  analysis,  and  prediction.  An 
alternative  to  simulation  is  the  use  of  highly  complex  stochastic  closed-form 
mathematical  models.  For  a  high-level  decision-maker,  taking  graduate-level  advanced 
math  and  statistics  courses  is  just  not  logical  or  practical.  A  well-informed  analyst  would 
use  all  available  tools  at  his  or  her  disposal  to  obtain  the  same  answer  the  easiest  and 
most  practical  way  possible.  And  in  all  cases,  when  modeled  correctly,  Monte  Carlo 
simulation  provides  similar  answers  to  the  more  mathematically  elegant  methods.  In 
addition,  there  are  many  real-life  applications  where  closed-form  models  do  not  exist 
and  the  only  recourse  is  to  apply  simulation  methods.  So,  what  exactly  is  Monte  Carlo 
simulation  and  how  does  it  work? 

Monte  Carlo  simulation  in  its  simplest  form  is  a  random  number  generator  that  is 
useful  for  forecasting,  estimation,  and  risk  analysis.  A  simulation  calculates  numerous 
scenarios  of  a  model  by  repeatedly  picking  values  from  a  user-predefined  probability 
distribution  for  the  uncertain  variables  and  using  those  values  for  the  model.  As  all  those 
scenarios  produce  associated  results  in  a  model,  each  scenario  can  have  a  forecast. 
Forecasts  are  events  (usually  with  formulas  or  functions)  that  you  define  as  important 
outputs  of  the  model. 

Think  of  the  Monte  Carlo  simulation  approach  as  picking  golf  balls  out  of  a  large 
basket  repeatedly  with  replacement.  The  size  and  shape  of  the  basket  depend  on  the 
distributional  input  assumption  (e.g.,  a  normal  distribution  with  a  mean  of  100  and  a 
standard  deviation  of  10,  versus  a  uniform  distribution  or  a  triangular  distribution)  where 
some  baskets  are  deeper  or  more  symmetrical  than  others,  allowing  certain  balls  to  be 
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pulled  out  more  frequently  than  others.  The  number  of  balls  pulled  repeatedly  depends 
on  the  number  of  trials  simulated.  Each  ball  is  indicative  of  an  event,  scenario,  or 
condition  that  can  occur.  For  a  large  model  with  multiple  related  assumptions,  imagine 
the  large  model  as  a  very  large  basket,  wherein  many  baby  baskets  reside.  Each  baby 
basket  has  its  own  set  of  colored  golf  balls  that  are  bouncing  around.  Sometimes  these 
baby  baskets  are  linked  with  each  other  (if  there  is  a  correlation  between  the  variables), 
forcing  the  golf  balls  to  bounce  in  tandem  whereas  in  other  uncorrelated  cases,  the  balls 
are  bouncing  independently  of  one  another.  The  balls  that  are  picked  each  time  from 
these  interactions  within  the  model  (the  large  basket)  are  tabulated  and  recorded, 
providing  a  forecast  output  result  of  the  simulation. 

Knowledge  Value  Added  Analysis 

As  the  U.S.  military  is  not  in  the  business  of  making  money,  referring  to  revenues 
throughout  this  paper  may  appear  to  be  a  misnomer.  For  nonprofit  organizations, 
especially  in  the  military,  we  require  a  Knowledge  Value  Added  (KVA)  model,  which  will 
provide  the  required  “benefits”  or  “revenue”  proxy  estimates  to  run  an  ROI  analysis.  ROI 
is  a  basic  productivity  ratio  with  revenue  in  the  numerator  and  cost  to  generate  the 
revenue  in  the  denominator  (actually  ROI  is  revenue-cost/cost).  KVA  generates  ROI 
estimates  by  developing  a  market  comparable  price  per  common  unit  of  output, 
multiplied  by  the  number  of  outputs,  to  achieve  a  total  revenue  estimate. 

KVA  is  a  methodology  whose  primary  purpose  is  to  describe  all  organizational 
outputs  in  common  units.  It  provides  a  means  to  compare  the  outputs  of  all  assets 
(human,  machine,  information  technology)  regardless  of  the  aggregated  outputs 
produced.  For  example,  the  purpose  of  a  military  process  may  be  to  gather  signal 
intelligence  or  plan  for  a  ship  alteration.  KVA  would  describe  the  outputs  of  both 
processes  in  common  units,  thus  making  their  performance  comparable. 

KVA  measures  the  value  provided  by  human  capital  and  IT  assets  by  analyzing 
an  organization,  process,  or  function  at  the  process  level.  It  provides  insights  into  each 
dollar  of  IT  investment  by  monetizing  the  outputs  of  all  assets,  including  intangible 
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assets  (e.g.,  such  as  that  produced  by  IT  and  humans).  By  capturing  the  value  of 
knowledge  embedded  in  an  organization’s  core  processes  (i.e.,  employees  and  IT), 

KVA  identifies  the  actual  cost  and  revenue  of  a  process,  product,  or  service.  Because 
KVA  identifies  every  process  required  to  produce  an  aggregated  output  in  terms  of  the 
historical  prices  and  costs  per  common  unit  of  output  of  those  processes,  unit  costs  and 
unit  prices  can  be  calculated.  The  methodology  has  been  applied  in  45  areas  within  the 
DoD,  from  flight  scheduling  applications  to  ship  maintenance  and  modernization 
processes. 

As  a  performance  tool,  the  KVA  methodology 

•  Compares  all  processes  in  terms  of  relative  productivity, 

•  Allocates  revenues  and  costs  to  common  units  of  output, 

•  Measures  value  added  by  IT  by  the  outputs  it  produces,  and 

•  Relates  outputs  to  the  cost  of  producing  those  outputs  in  common  units. 

Based  on  the  tenets  of  complexity  theory,  KVA  assumes  that  humans  and 
technology  in  organizations  add  value  by  taking  inputs  and  changing  them  (measured  in 
units  of  complexity)  into  outputs  through  core  processes.  The  amount  of  change  an 
asset  within  a  process  produces  can  be  a  measure  of  value  or  benefit.  The  additional 
assumptions  in  KVA  include  the  following: 

•  Describing  all  process  outputs  in  common  units  (e.g.,  using  a  knowledge 
metaphor  for  the  descriptive  language  in  terms  of  the  time  it  takes  an  average 
employee  to  learn  how  to  produce  the  outputs)  allows  historical  revenue  and  cost 
data  to  be  assigned  to  those  processes  historically. 

•  All  outputs  can  be  described  in  terms  of  the  time  required  to  learn  how  to 
produce  them. 

•  Learning  Time,  a  surrogate  for  procedural  knowledge  required  to  produce 
process  outputs,  is  measured  in  common  units  of  time.  Consequently,  Units  of 
Learning  Time  =  Common  Units  of  Output  (K). 
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•  Common  unit  of  output  makes  it  possible  to  compare  all  outputs  in  terms  of  cost 
per  unit  as  well  as  price  per  unit,  because  revenue  can  now  be  assigned  at  the 
suborganizational  level. 

•  Once  cost  and  revenue  streams  have  been  assigned  to  suborganizational 
outputs,  normal  accounting  and  financial  performance  and  profitability  metrics 
can  be  applied  (Rodgers  &  Housel,  2006;  Pavlou,  Housel,  Rodgers,  &  Jansen, 
2005;  Housel  &  Kanevsky,  1995a). 

Describing  processes  in  common  units  also  permits  market  comparable  data  to 
be  generated,  which  is  particularly  important  for  nonprofits  like  the  U.S.  military.  Using  a 
market  comparables  approach,  data  from  the  commercial  sector  can  be  used  to 
estimate  price  per  common  unit,  allowing  for  revenue  estimates  of  process  outputs  for 
nonprofits.  This  approach  also  provides  a  common  units  basis  to  define  benefit  streams 
regardless  of  the  process  analyzed. 

KVA  differs  from  other  nonprofit  ROI  models  because  it  allows  for  revenue 
estimates,  enabling  the  use  of  traditional  accounting,  financial  performance,  and 
profitability  measures  at  the  suborganizational  level.  KVA  can  rank  processes  by  the 
degree  to  which  they  add  value  to  the  organization  or  its  outputs.  This  ranking  assists 
decision-makers  in  identifying  how  much  value  processes  add.  Value  is  quantified  in 
two  key  metrics:  Return  on  Knowledge  (ROK:  revenue/cost)  and  ROI  (revenue- 
investment  cost/investment  cost).  The  outputs  from  a  KVA  analysis  become  the  input 
into  the  ROI  models  and  real  options  analysis.  By  tracking  the  historical  volatility  of  price 
and  cost  per  unit  as  well  as  ROI,  it  is  possible  to  establish  risk  (as  compared  to 
uncertainty)  distributions,  which  is  important  for  accurately  estimating  the  value  of  real 
options. 

The  KVA  method  has  been  applied  to  numerous  military  core  processes  across 
the  Services.  KVA  research  has  more  recently  provided  a  means  for  simplifying  real 
options  analysis  for  DoD  processes.  Current  KVA  research  will  provide  a  library  of 
market-comparable  price  and  cost-per-unit  of  output  estimates.  This  research  will 
enable  a  more  stable  basis  for  comparisons  of  performance  across  core  processes. 
These  data  also  provide  a  means  to  establish  risk  distribution  profiles  for  Integrated 
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Risk  Management  approaches  such  as  real  options,  and  KVA  currently  is  being  linked 
directly  to  the  Real  Options  Super  Lattice  Solver  and  Risk  Simulator  software  for  rapid 
adjustments  to  real  options  valuation  projections. 

Strategic  Real  Options  Analysis 

Suppose  you  are  driving  from  point  A  to  point  B,  and  you  only  have  or  know  one 
way  to  get  there,  a  straight  route.  Further  suppose  that  there  is  a  lot  of  uncertainty  as  to 
what  traffic  conditions  are  like  further  down  the  road,  and  you  risk  being  stuck  in  traffic, 
and  there’s  a  50%  chance  that  will  occur.  Simulation  will  provide  you  the  50%  figure. 

But  so  what?  Knowing  that  half  the  time  you  will  get  stuck  in  traffic  is  valuable 
information,  but  the  question  now  is,  so  what?  Especially  if  you  have  to  get  to  point  B  no 
matter  what.  However,  if  you  had  several  alternate  routes  to  get  to  point  B,  you  could 
still  drive  the  straight  route,  but  if  you  hit  traffic,  you  could  make  a  left,  right,  or  U-turn,  to 
get  around  congestion,  mitigating  the  risk,  and  getting  you  to  point  B  faster  and  safer; 
that  is,  you  have  options.  So,  how  much  is  such  a  strategic  road  map  or  global 
positioning  satellite  map  worth  to  you?  In  military  situations  with  high  risk,  real  options 
can  help  you  create  strategies  to  mitigate  these  risks.  In  fact,  businesses  and  the 
military  have  been  doing  real  options  for  hundreds  of  years  without  realizing  it.  For 
instance,  in  the  military,  we  call  it  courses  of  action  or  analysis  of  alternatives — do  we 
take  Hill  A  so  that  it  provides  us  the  option  and  ability  to  take  Hill  B  and  Valley  C,  or  how 
should  we  take  Valley  C,  or  do  we  avoid  taking  Valley  C  altogether,  and  so  forth.  A 
piece  that  is  missing  is  the  more  formal  structure  and  subsequent  analytics  that  real 
options  analysis  provides.  Using  real  options  analysis,  we  can  quantify  and  value  each 
strategic  pathway,  and  frame  strategies  that  will  hedge  or  mitigate,  and  sometimes  take 
advantage  of,  risk. 

In  the  past,  corporate  investment  decisions  were  cut-and-dried.  Buy  a  new 
machine  that  is  more  efficient,  make  more  products  costing  a  certain  amount,  and  if  the 
benefits  outweigh  the  costs,  execute  the  investment.  Hire  a  larger  pool  of  sales 
associates,  expand  the  current  geographical  area,  and  if  the  marginal  increase  in 
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forecast  sales  revenues  exceeds  the  additional  salary  and  implementation  costs,  start 
hiring.  Need  a  new  manufacturing  plant?  Show  that  the  construction  costs  can  be 
recouped  quickly  and  easily  by  the  increase  in  revenues  it  will  generate  through  new 
and  more  improved  products,  and  the  initiative  is  approved.  However,  real-life 
conditions  are  a  lot  more  complicated.  Your  firm  decides  to  go  with  a  more  automated 
3D  PDF  software  and  Logistics  Team  Center  environment,  but  multiple  strategic  paths 
exist.  Which  path  do  you  choose?  What  are  the  options  that  you  have?  If  you  choose 
the  wrong  path,  how  do  you  get  back  on  the  right  track?  How  do  you  value  and  prioritize 
the  paths  that  exist?  You  are  a  venture  capitalist  firm  with  multiple  business  plans  to 
consider.  How  do  you  value  a  start-up  firm  with  no  proven  track  record?  How  do  you 
structure  a  mutually  beneficial  investment  deal?  What  is  the  optimal  timing  to  a  second 
or  third  round  of  financing? 

Real  options  are  useful  not  only  in  valuing  a  firm,  asset,  or  investment  decision 
through  its  strategic  business  options  but  also  as  a  strategic  business  tool  in  capital 
investment  acquisition  decisions.  For  instance,  should  the  military  invest  millions  in  a 
new  open  architecture  initiative,  and,  if  so,  what  are  the  values  of  the  various  strategies 
such  an  investment  would  enable,  and  how  do  we  proceed?  How  does  the  military 
choose  among  several  seemingly  cashless,  costly,  and  unprofitable  information- 
technology  infrastructure  projects?  Should  it  indulge  its  billions  in  a  risky  research  and 
development  initiative?  The  consequences  of  a  wrong  decision  can  be  disastrous  and 
lives  could  be  at  stake.  In  a  traditional  analysis,  these  questions  cannot  be  answered 
with  any  certainty.  In  fact,  some  of  the  answers  generated  through  the  use  of  the 
traditional  analysis  are  flawed  because  the  model  assumes  a  static,  one-time  decision¬ 
making  process,  while  the  real  options  approach  takes  into  consideration  the  strategic 
options  certain  projects  create  under  uncertainty  and  a  decision-maker’s  flexibility  in 
exercising  or  abandoning  these  options  at  different  points  in  time,  when  the  level  of 
uncertainty  has  decreased  or  has  become  known  over  time. 

Real  options  analysis  can  be  used  to  frame  strategies  to  mitigate  risk,  to  value 
and  find  the  optimal  strategic  pathway  to  pursue,  and  to  generate  options  to  enhance 
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the  value  of  the  project  while  managing  risks.  Sample  options  include  the  option  to 
expand,  contract,  abandon,  or  sequentially  compound  options  (phased  stage-gate 
options,  options  to  wait  and  defer  investments,  proof  of  concept  stages,  milestone 
development,  and  research  and  development  initiatives).  Some  sample  applications  in 
the  military  include  applications  of  real  options  to  acquisitions,  Spiral  Development,  and 
various  organizational  configurations,  as  well  as  the  importance  of  how  Integrated  and 
Open  Architectures  become  real  options  multipliers.  Under  OMB  Circular  A-76  (Office  of 
Budget  Management  [OMB],  2003),  comparisons  using  real  options  analysis  could  be 
applied  to  enhance  outsourcing  comparisons  between  the  Government’s  Most  Efficient 
Organization  (MEO)  and  private  sector  alternatives.  Real  options  can  be  used 
throughout  JCIDS  requirements  generation  and  the  Defense  Acquisition  System,  for 
example,  DOTMLPF  versus  New  Program/Service  solution,  Joint  Integration,  Analysis 
of  Material  Alternatives  (AMA),  Analysis  of  Alternatives  (AoA),  and  Spiral  Development. 
Many  other  applications  exist  in  military  decision  analysis  and  portfolios. 

Real  Options:  A  Quick  Peek  Behind  the  Scenes 

Real  options  analysis  will  be  performed  to  determine  the  prospective  value  of  the 
basic  options  over  a  multiyear  period  using  KVA  data  as  a  platform.  The  strategic  real 
options  analysis  is  solved  employing  various  methodologies,  including  the  use  of 
binomial  lattices  with  a  market-replicating  portfolios  approach,  and  backed  up  using  a 
modified  closed-form  sequential  compound  option  model.  The  value  of  a  compound 
option  is  based  on  the  value  of  another  option.  That  is,  the  underlying  variable  for  the 
compound  option  is  another  option,  and  the  compound  option  can  be  either  sequential 
in  nature  or  simultaneous.  Solving  such  a  model  requires  programming  capabilities. 

This  subsection  is  meant  as  a  quick  peek  into  the  math  underlying  a  very  basic  closed- 
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form  compound  option.21  This  section  is  only  a  preview  of  the  detailed  modeling 
techniques  used  in  the  current  analysis  and  should  not  be  assumed  to  be  the  final  word. 

For  instance,  we  first  start  by  solving  for  the  critical  value  of  I,  an  iterative 
component  in  the  model  using 
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We  then  solve  recursively  for  the  value  /  and  input  it  into  the  model: 
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\n(S  /  Xl)  +  (r-q  + a2  /2)T2 


a 


ln(5 / I)  +  (r-q  +  a2  / 2% 


-X2e~rtl<t> 


aJt 


\n(S  /  I)  +  (r-q  +  a2 /2)t, 


ti  , 


1 5  4~t\ !  T2 


-aJt, 


The  model  is  then  applied  to  a  sequential  problem  where  future  phase  options 
depend  on  previous  phase  options  (e.g.,  Phase  II  depends  on  Phase  I’s  successful 
implementation). 


21  We  recommend  reviewing  Real  Options  Analysis:  Tools  and  Techniques,  Second  Edition,  by 
Johnathan  Mun  (2006)  for  more  hands-on  details  and  modeling  techniques  used  in  the  analysis. 
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Definitions  of  Variables 


S  present  value  of  future  cash  flows  ($) 
r  risk-free  rate  (%) 

cr  volatility  (%) 

0  cumulative  standard-normal 

q  continuous  dividend  payout  (%) 

/  critical  value  solved  recursively 

Q  cumulative  bivariate-normal 

Xi  strike  for  the  underlying  ($) 

X2  strike  for  the  option  on  the  option  ($) 

ti  expiration  date  for  the  option  on  the  option 
T2  expiration  date  for  the  underlying  option 

The  preceding  closed-form  differential  equation  models  are  then  verified  using 
the  risk-neutral  market-replicating  portfolio  approach,  assuming  a  sequential  compound 
option.  In  solving  the  market-replicating  approach,  we  use  the  following  functional  forms 
(Mun,  2006): 


Hedge  ratio  (h): 


k; 


C  -C 

up  down 

s  -  s 

u  up  down 


Debt  load  (D): 

=Si{hi_l)-Ci 

Call  value  (C)  at  node  /: 

c,  =S,(h,)-Dle-*w 


Risk-adjusted  probability  (q): 
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obtained  assuming 


c  _  e 

°i- 1  °down 

C^i  s  -  s 

^  up  u  down 

Si-i  =  A  +  0  -  q,)s* 


This  means  that 

^i- 1  ^1 up  ^down  down  gp^j  ^ do wn  ]  ^i- 1  ^ down 

C  -  9 

Ji-1  ^down 


so  we  get  qt  = 


5  -S'. 

up  down 


Portfolio  Optimization 


In  most  decisions,  there  are  variables  over  which  leadership  has  control,  such  as 
how  much  to  establish  supply  lines,  modernize  a  ship,  use  network  centricity  to  gather 
intelligence,  and  so  on.  Similarly,  business  leaders  have  options  in  what  they  charge  for 
a  product  or  how  much  to  invest  in  a  project  or  which  projects  they  should  choose  in  a 
portfolio  when  they  are  constrained  by  budgets  or  resources.  These  decisions  could 
also  include  allocating  financial  resources,  building  or  expanding  facilities,  managing 
inventories,  and  determining  product-mix  strategies.  Such  decisions  might  involve 
thousands  or  millions  of  potential  alternatives.  Considering  and  evaluating  each  of  them 
would  be  impractical  or  even  impossible.  These  controlled  variables  are  called  decision 
variables.  Finding  the  optimal  values  for  decision  variables  can  make  the  difference 
between  reaching  an  important  goal  and  missing  that  goal.  An  optimization  model  can 
provide  valuable  assistance  in  incorporating  relevant  variables  when  analyzing 
decisions,  and  finding  the  best  solutions  for  making  decisions.  Optimization  models 
often  provide  insights  that  intuition  alone  cannot.  An  optimization  model  has  three  major 
elements:  decision  variables,  constraints,  and  an  objective.  In  short,  the  optimization 
methodology  finds  the  best  combination  or  permutation  of  decision  variables  (e.g.,  best 
way  to  deploy  troops,  build  ships,  which  projects  to  execute)  in  every  conceivable  way 
such  that  the  objective  is  maximized  (e.g.,  strategic  value,  enemy  assets  destroyed, 
return  on  investment)  or  minimized  (e.g.,  risk  and  costs)  while  still  satisfying  the 
constraints  (e.g.,  time,  budget,  and  resources). 
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Obtaining  optimal  values  generally  requires  that  you  search  in  an  iterative  or  ad 
hoc  fashion.  This  search  involves  running  one  iteration  for  an  initial  set  of  values, 
analyzing  the  results,  changing  one  or  more  values,  rerunning  the  model,  and  repeating 
the  process  until  you  find  a  satisfactory  solution.  This  process  can  be  very  tedious  and 
time  consuming  even  for  small  models,  and  often  it  is  not  clear  how  to  adjust  the  values 
from  one  iteration  to  the  next.  A  more  rigorous  method  systematically  enumerates  all 
possible  alternatives.  This  approach  guarantees  optimal  solutions  if  the  model  is 
correctly  specified.  Suppose  that  an  optimization  model  depends  on  only  two  decision 
variables.  If  each  variable  has  10  possible  values,  trying  each  combination  requires  100 
iterations  (102  alternatives).  If  each  iteration  is  very  short  (e.g.,  2  seconds),  then  the 
entire  process  could  be  done  in  approximately  three  minutes  of  computer  time. 
However,  instead  of  two  decision  variables,  consider  six,  then  consider  that  trying  all 
combinations  requires  1,000,000  iterations  (106  alternatives).  It  is  easily  possible  for 
complete  enumeration  to  take  many  years  to  carry  out.  Therefore,  optimization  has 
always  been  a  fantasy  until  now;  with  the  advent  of  sophisticated  software  and 
computing  power,  coupled  with  smart  heuristics  and  algorithms,  such  analyses  can  be 
done  within  minutes. 
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Start 


Strategy  A 


Divide  the  development  into 
multiple  milestones,  stage-gating 
the  developmentand  mitigating 
the  risk 


Phase  I 


Milestone  1 


Exit 


Do  nothing 


Strategy  B 


Apply  a  quick  proof  of  concept 
of  the  technology,  replicate  and 
scale  to  larger  force-wide 
deployment.  If  POC  fails 
abandon  project  and  reduce 
future  risks.  Outsource  or 
obtain  off-the-shelf  platforms  if 
successful 


*  POC 


Small  Scale 
Proof  of  Concept 


*  Exit 


Do  nothing 


Strategy  C  — 


Purchase  technol  ogy  rather  than 
developing  it  in-house.  If 
a  p  plication  fails,  abandon  and 
sell  the  company's  intellectual 
property  and  technology.  If 
successful,  find  other 
applicati  ons  for  technology. 


Buy 


Purchase 

technology 


Exit 


Do  nothing 


Milestone  2 


Phase  III 


Milestone  3 

Phase  ll 

- >► 

Exit 

>  Phase  IV 


Milestone  4 


Exit 


Stop  after  Phase  III 


Stop  after  Phase II 


Exit 


Stop  after  Phase  I 


Phase  II 


Larger  Deployment 


Contract 


Outsourcing 


Exit 


Exit 


Internal  development  rather 
than  off-the-shelf  applications 


Abandon 


Expand 


R&Dn  ew  t  ethnology 
expand  into  other 
application  areas 


Exit 


Sell  IP  and  technology, 
abandon  project 


Figure  A.5.  Example  of  Real  Options  Framing 

Figures  A.6,  A.7,  and  A.8  illustrate  a  sample  portfolio  analysis  where  in  the  first 
case,  there  are  20  total  projects  to  choose  from  (if  all  projects  were  executed,  it  would 
cost  $10.2  billion)  and  where  each  project  has  its  own  returns  on  investment  or  benefits 
measure,  cost,  strategic  ranking,  comprehensive,  and  tactical  and  total  military  scores 
(these  were  obtained  from  field  commanders  through  the  Delphi  method  to  elicit  their 
thoughts  about  how  strategic  a  particular  project  or  initiative  will  be,  and  so  forth).  The 
constraints  are  full-time  equivalence  resources,  budget,  and  strategic  score.  In  other 
words,  there  are  20  projects  or  initiatives  to  choose  from,  where  we  want  to  select  the 
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top  10,  subject  to  having  enough  money  to  pay  for  them  and  the  people  to  do  the  work, 
and  yet  be  the  most  strategic  portfolio  possible.22  All  the  while,  Monte  Carlo  simulation, 
real  options,  and  forecasting  methodologies  are  applied  in  the  optimization  model  (e.g., 
each  project’s  values  shown  in  Figure  A.6  are  linked  from  its  own  large  model  with 
simulation  and  forecasting  methodologies  applied,  and  the  best  strategy  for  each  project 
is  chosen  using  real  options  analysis,  or  perhaps  the  projects  shown  are  nested  within 
one  another;  for  instance,  you  cannot  exercise  Project  2  unless  you  execute  Project  1, 
but  you  can  exercise  Project  1  without  having  to  do  Project  2,  and  so  forth).  The  results 
are  shown  in  Figure  A.6. 

Figure  A.7  shows  the  optimization  process  done  in  series,  while  relaxing  some  of 
the  constraints.  For  instance,  what  would  be  the  best  portfolio  and  the  strategic  outcome 
if  a  budget  of  $3.8  billion  was  imposed?  What  if  it  was  increased  to  $4.8  billion,  $5.8 
billion,  and  so  forth?  The  efficient  frontiers  depicted  in  Figure  A.7  illustrate  the  best 
combination  and  permutation  of  projects  in  the  optimal  portfolio.  Each  point  on  the 
frontier  is  a  portfolio  of  various  combinations  of  projects  that  provides  the  best  allocation 
possible  given  the  requirements  and  constraints.  Finally,  Figure  A.8  shows  the  top  10 
projects  that  were  chosen  and  how  the  total  budget  is  best  and  most  optimally  allocated 
to  provide  the  best  and  most  well-balanced  portfolio. 


22  There  are  2  x  1018  possible  permutations  for  this  problem,  and  if  tested  by  hand,  the  calculation  would 
take  years  to  complete.  Using  Risk  Simulator,  the  problem  is  solved  in  about  5  seconds,  or  several 
minutes  if  Monte  Carlo  simulation  and  real  options  are  incorporated  in  the  analysis. 
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Project 

Name 

ENPU 

Benefits 

Cost 

Strategy 

Ranking 

Project  1 

$458.00 

$150.76 

$1,732.44 

1.20 

Project  2 

$1,954.00 

$245.00 

$859.00 

9.80 

Project  3 

$1,599.00 

$458.00 

$1,845.00 

9.70 

Project  4 

$2,251.00 

$529.00 

$1,645.00 

4.50 

Project  5 

$849.00 

$564.00 

$458.00 

10.90 

Project  6 

$758.00 

$*35.00 

$52.00 

7.40 

Project  7 

$2,845.00 

$311.00 

$758.00 

19.80 

Project  8 

$1,235.00 

$754.00 

$115.00 

7.50 

Project  9 

$1,945.00 

$198.00 

$125.00 

10.80 

Project  1 0 

$2,250.00 

$785.00 

$45E  CD 

8.50 

Project  11 

$549.00 

$35.00 

$45.00 

4.80 

Project  12 

$525.00 

$75.00 

$105.00 

5.90 

Project  1 3 

$516.00 

$451.00 

$48.00 

2.80 

Project  14 

$499.00 

$458.00 

$351.00 

9.40 

Project  15 

$859.00 

$125.00 

$421.00 

6.50 

Project  1 6 

$884.00 

$458.00 

$124.00 

3.90 

Project  17 

$956.00 

$124.00 

$521.00 

15.40 

Project  18 

$854.00 

$164.00 

$512.00 

21.00 

Project  1 9 

$195.00 

$45.00 

$5.00 

1.20 

Project  20 

$210.00 

$85.00 

$21.00 

1.00 

Total  $14,185.00  $0,784.00  00.00 

ProWRank  $143.28 

profiTScore  $470,235.60  Maximize  <=$3800  <=100 


Return  to 

Profitability 

Selection 

Comprehensive 

Tactical 

FTE 

Rank  Ratio 

index 

Score 

Score 

Resources 

381.67 

1.09 

0 

8.10 

2.31 

1.20 

199.39 

1.29 

1 

1.27 

483 

2.50 

164.85 

1.25 

0 

9.88 

4.75 

3.60 

500.22 

1.32 

0 

8.83 

161 

4.50 

77.89 

2.23 

0 

5.02 

625 

5.50 

102.43 

3.60 

1 

3.64 

5  79 

9.20 

143.69 

1.41 

1 

5.27 

6.47 

12.50 

16467 

7.56 

1 

9.80 

7.16 

5.30 

180.09 

2.58 

1 

5.68 

239 

6.30 

264.71 

2.71 

1 

829 

4.41 

4.50 

114.38 

1.78 

0 

7.52 

465 

4.90 

88.98 

1.71 

0 

5.54 

509 

5.20 

184.29 

10.40 

0 

2.51 

2.17 

4.60 

53.09 

2.30 

1 

9.41 

9.49 

9.90 

132.15 

1.30 

1 

6.91 

962 

7.20 

226.67 

4.69 

1 

7.06 

9.98 

7.50 

62.08 

1.24 

1 

1.25 

2.50 

8.60 

40.67 

1.32 

0 

3.09 

2.90 

4.30 

162.50 

10.00 

0 

5.25 

1.22 

4.10 

210.00 

5.05 

0 

2.01 

4.06 

5.20 

10 

5ft  58 

62.64 

73.50 

x<=10  <=80 


Figure  A.6.  Portfolio  Optimization  and  Allocation 
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MS titary 
Score 

1.98 

1.76 

2.77 
207 
2.94 
326 
4  04 
363 
216 
2.67 
2.75 
269 
166 
485 
4.25 
4.46 
207 
1.70 
1  86 
2.50 

33.15 


Military 

Tactical 

Comprehensive 

Allowed 

R  Of -RANK 

Budget 

Score 

Score 

Score 

Projects 

Objective 

$3,800.00 

33.15 

62.64 

58.58 

10 

$470,235.60 

$4,800.00 

36.33 

68.85 

66.86 

11 

$521,645.92 

$5,800.00 

38.40 

7046 

7569 

12 

$623,557.79 

$0,800.00 

39.94 

72.14 

82.31 

13 

$659,947.99 

$7,800.00 

39.78 

70.05 

88.54 

14 

$878,279.81 

Military  Score 


Comprehensive  Score 


Figure  A.7.  Efficient  Frontiers  of  Portfolios 


ASSET  ALLOCATION  OPTIMIZATION  MODEL 


Asset  Class 

Annualized 

Volatility 

Allocation 

Required 

Required 

Return  to 

Returns 

Risk 

Return  to 

Allocation 

Description 

Returns 

Risk 

Weights 

Minimum 

Allocation 

Maximum 

Allocation 

Risk  Ratio 

Ranking 

(Hi-Lo) 

Ranking 

(Lo-Hi) 

Risk  Ranking 
(Hi-Lo) 

Ranking 

(Hi-Lo) 

Selected  Project  1 

10.50% 

12.38% 

11.10% 

5.00% 

35.00% 

0.8483 

9 

2 

7 

4 

Selected  Project  2 

11.12% 

16.36% 

6.74% 

5.00% 

35.00% 

0.6799 

7 

8 

10 

10 

Selected  Project  3 

11.77% 

15.81% 

7.63% 

5.00% 

35.00% 

0.7445 

6 

7 

9 

9 

Selected  Project  4 

10  77% 

12  33% 

1 1  49% 

5  nn% 

35  nn% 

fl  R73R 

R 

1 

5 

3 

Selected  Project  5 

13.49% 

13.35% 

12.26% 

5.00% 

35.00% 

1.0102 

5 

4 

2 

2 

Selected  Project  6 

14.24% 

14.53% 

10.94% 

5.00% 

35.00% 

0.9800 

3 

6 

3 

5 

Selected  Project  7 

15.60% 

14.30% 

12.36% 

5.00% 

35.00% 

1.0908 

1 

5 

1 

1 

Selected  Project  8 

14.95% 

16.64% 

8.75% 

5.00% 

35.00% 

0.8983 

2 

10 

4 

7 

Selected  Project  9 

14.15% 

16.56% 

8.36% 

5.00% 

35.00% 

0.8545 

4 

9 

6 

8 

Selected  Project  10 

10.08% 

12.55% 

10.37% 

5.00% 

35.00% 

0.8027 

10 

3 

8 

6 

Portfolio  Total 

12.1210% 

4.54%  | 

100.00% 

1 

Return  to  Risk  Ratio  \  2.8021 


Figure  A.8.  Portfolio  Optimization  (Continuous  Allocation  of  Funds) 
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Integrated  Risk  Management  Framework 


We  are  now  able  to  put  all  the  pieces  together  into  an  integrated  risk 
management  framework  and  see  how  these  different  techniques  are  related  in  a  risk 
analysis  and  risk  management  context.  This  framework  comprises  eight  distinct  phases 
of  a  successful  and  comprehensive  risk  analysis  implementation,  going  from  a 
qualitative  management  screening  process  to  creating  clear  and  concise  reports  for 
management.  The  process  was  developed  by  the  author  (Mun)  based  on  previous 
successful  implementations  of  risk  analysis,  forecasting,  real  options,  KVA  cash-flow 
estimates,  valuation,  and  optimization  projects  both  in  the  consulting  arena  and  for 
industry-specific  problems.  These  phases  can  be  performed  either  in  isolation  or 
together  in  sequence  for  a  more  robust  integrated  analysis. 

Figure  A.9  shows  the  integrated  risk  management  process  up  close.  We  can 
segregate  the  process  into  the  following  eight  simple  steps: 

1)  Qualitative  management  screening 

2)  Time-series  and  regression  forecasting 

3)  Base  case  KVA  and  net  present  value  analysis 

4)  Monte  Carlo  simulation 

5)  Real  options  problem  framing 

6)  Real  options  modeling  and  analysis 

7)  Portfolio  and  resource  optimization 

8)  Reporting  and  update  analysis 
1.  Qualitative  Management  Screening 

Qualitative  management  screening  is  the  first  step  in  any  integrated  risk 
management  process.  Decision-makers  have  to  decide  which  projects,  assets, 
initiatives,  or  strategies  are  viable  for  further  analysis,  in  accordance  with  the 
organization’s  mission,  vision,  goal,  or  overall  business  strategy.  The  organization’s 
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mission,  vision,  goal,  or  overall  business  strategy  may  include  strategies  and  tactics, 
and  competitive  advantage,  technical,  acquisition,  growth,  synergistic,  or  global  threat 
issues.  That  is,  the  initial  list  of  projects  should  be  qualified  in  terms  of  meeting  the 
leadership’s  agenda.  Often  the  most  valuable  insight  is  created  as  leaders  frame  the 
complete  problem  to  be  resolved.  This  is  where  the  various  risks  to  the  organization  are 
identified  and  fleshed  out. 

2.  Time-Series  and  Regression  Forecasting 

The  future  is  then  forecasted  using  time-series  analysis,  stochastic  forecasting, 
or  multivariate  regression  analysis  if  historical  or  comparable  data  exist.  Otherwise, 
other  qualitative  forecasting  methods  may  be  used  (subjective  guesses,  growth  rate 
assumptions,  expert  opinions,  Delphi  method,  and  so  forth).23 

3.  Base  Case  KVA  and  Net  Present  Value  Analysis 

For  each  project  that  passes  the  initial  qualitative  screens,  a  KVA-based 
discounted  cash  flow  model  is  created.  This  model  serves  as  the  base  case  analysis 
where  a  net  present  value  and  ROI  are  calculated  for  each  project,  using  the  forecasted 
values  in  the  previous  step.  This  step  also  applies  if  only  a  single  project  is  under 
evaluation.  This  net  present  value  is  calculated  with  the  traditional  approach  of  using  the 
forecast  revenues  and  costs,  and  discounting  the  net  of  these  revenues  and  costs  at  an 
appropriate  risk-adjusted  rate.  The  ROI  and  other  financial  metrics  are  generated  here. 

4.  Monte  Carlo  Simulation24 

Because  the  static  discounted  cash  flow  produces  only  a  single-point  estimate 
result,  there  is  oftentimes  little  confidence  in  its  accuracy  given  that  future  events  that 


23  See  Chapters  8  and  9  of  Modeling  Risk  (Wiley,  2006)  by  Dr.  Johnathan  Mun  for  details  on  forecasting 
and  using  the  author’s  Risk  Simulator  software  to  run  time-series  analysis,  extrapolation,  stochastic 
process,  ARIMA,  and  regression  forecasts. 

24  See  Chapters  4  and  5  of  Modeling  Risk  (Wiley,  2006)  by  Dr.  Johnathan  Mun  for  details  on  running 
Monte  Carlo  simulation  using  the  author’s  Risk  Simulator  software. 
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affect  forecast  cash  flows  are  highly  uncertain.  To  better  estimate  the  actual  value  of  a 
particular  project,  Monte  Carlo  simulation  should  be  employed  next.  Usually,  a 
sensitivity  analysis  is  first  performed  on  the  discounted  cash  flow  model;  that  is,  setting 
the  net  present  value  or  ROI  as  the  resulting  variable,  we  can  change  each  of  its 
precedent  variables  and  note  the  change  in  the  resulting  variable.  Precedent  variables 
include  revenues,  costs,  tax  rates,  discount  rates,  capital  expenditures,  depreciation, 
and  so  forth,  which  ultimately  flow  through  the  model  to  affect  the  net  present  value  or 
ROI  figure.  By  tracing  back  all  these  precedent  variables,  we  can  change  each  one  by  a 
preset  amount  and  see  the  effect  on  the  resulting  net  present  value.  A  graphical 
representation  can  then  be  created  in  Risk  Simulator,  which  is  often  called  a  tornado 
chart  because  of  its  shape,  where  the  most  sensitive  precedent  variables  are  listed  first, 
in  descending  order  of  magnitude.  Armed  with  this  information,  the  analyst  can  then 
decide  which  key  variables  are  highly  uncertain  in  the  future  and  which  are 
deterministic.  The  uncertain  key  variables  that  drive  the  net  present  value  and,  hence, 
the  decision  are  called  critical  success  drivers.  These  critical  success  drivers  are  prime 
candidates  for  Monte  Carlo  simulation.  Because  some  of  these  critical  success  drivers 
may  be  correlated,  a  correlated  and  multidimensional  Monte  Carlo  simulation  may  be 
required.  Typically,  these  correlations  can  be  obtained  through  historical  data.  Running 
correlated  simulations  provides  a  much  closer  approximation  to  the  variables’  real-life 
behaviors. 

5.  Real  Options  Problem  Framing25 

The  question  now  is  that  after  quantifying  risks  in  the  previous  step,  what  next? 
The  risk  information  obtained  somehow  needs  to  be  converted  into  actionable 
intelligence.  Just  because  risk  has  been  quantified  to  be  such  and  such  using  Monte 
Carlo  simulation,  so  what  and  what  do  we  do  about  it?  The  answer  is  to  use  real  options 
analysis  to  hedge  these  risks,  to  value  these  risks,  and  to  position  yourself  to  take 


25  See  Real  Options  Analysis:  Tools  and  Techniques,  Second  Edition  (Wiley,  2005)  by  Dr.  Johnathan  Mun 
for  more  technical  details  on  framing  and  solving  real  options  problems. 
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advantage  of  the  risks.  The  first  step  in  real  options  is  to  generate  a  strategic  map 
through  the  process  of  framing  the  problem.  Based  on  the  overall  problem  identification 
occurring  during  the  initial  qualitative  management  screening  process,  certain  strategic 
optionalities  would  have  become  apparent  for  each  particular  project.  The  strategic 
optionalities  may  include,  among  other  things,  the  option  to  expand,  contract,  abandon, 
switch,  choose,  and  so  forth.  Based  on  the  identification  of  strategic  optionalities  that 
exist  for  each  project  or  at  each  stage  of  the  project,  the  analyst  can  then  choose  from  a 
list  of  options  to  analyze  in  more  detail.  Real  options  are  added  to  the  projects  to  hedge 
downside  risks  and  to  take  advantage  of  upside  swings. 

6.  Real  Options  Modeling  and  Analysis 

Through  the  use  of  Monte  Carlo  simulation,  the  resulting  stochastic  discounted 
cash  flow  model  will  have  a  distribution  of  values.  Thus,  simulation  models,  analyzes, 
and  quantifies  the  various  risks  and  uncertainties  of  each  project.  The  result  is  a 
distribution  of  the  NPVs  and  the  project’s  volatility.  In  real  options,  we  assume  that  the 
underlying  variable  is  the  future  profitability  of  the  project,  which  is  the  future  cash  flow 
series.  An  implied  volatility  of  the  future  free  cash  flow  or  underlying  variable  can  be 
calculated  through  the  results  of  a  Monte  Carlo  simulation  previously  performed. 

Usually,  the  volatility  is  measured  as  the  standard  deviation  of  the  logarithmic  returns  on 
the  free  cash  flow  stream.  In  addition,  the  present  value  of  future  cash  flows  for  the 
base  case  discounted  cash  flow  model  is  used  as  the  initial  underlying  asset  value  in 
real  options  modeling.  Using  these  inputs,  real  options  analysis  is  performed  to  obtain 
the  projects’  strategic  option  values. 

7.  Portfolio  and  Resource  Optimization26 

Portfolio  optimization  is  an  optional  step  in  the  analysis.  If  the  analysis  is  done  on 
multiple  projects,  decision-makers  should  view  the  results  as  a  portfolio  of  rolled-up 


26  See  Chapters  1 0  and  1 1  of  Modeling  Risk  (Wiley,  2006)  by  Dr.  Johnathan  Mun  for  details  on  using  Risk 
Simulator  to  perform  portfolio  optimization. 
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projects  because  the  projects  are,  in  most  cases,  correlated  with  one  another,  and 
viewing  them  individually  will  not  present  the  true  picture.  As  organizations  do  not  have 
only  single  projects,  portfolio  optimization  is  crucial.  Given  that  certain  projects  are 
related  to  others,  there  are  opportunities  for  hedging  and  diversifying  risks  through  a 
portfolio.  Because  firms  have  limited  budgets  and  time  and  resource  constraints,  while 
at  the  same  time  having  requirements  for  certain  overall  levels  of  returns,  risk 
tolerances,  and  so  forth,  portfolio  optimization  takes  into  account  all  these  to  create  an 
optimal  portfolio  mix.  The  optimization  analysis  will  provide  the  optimal  allocation  of 
investments  across  multiple  projects. 

8.  Reporting  and  Update  Analysis 

The  analysis  is  not  complete  until  reports  can  be  generated.  Not  only  are  results 
presented,  but  the  process  should  also  be  shown.  Clear,  concise,  and  precise 
explanations  transform  a  difficult  black-box  set  of  analytics  into  transparent  steps. 
Decision-makers  will  never  accept  results  coming  from  black  boxes  if  they  do  not 
understand  where  the  assumptions  or  data  originate  and  what  types  of  mathematical  or 
analytical  massaging  takes  place.  Risk  analysis  assumes  that  the  future  is  uncertain 
and  that  decision-makers  have  the  right  to  make  midcourse  corrections  when  these 
uncertainties  become  resolved  or  risks  become  known;  the  analysis  is  usually  done 
ahead  of  time  and,  thus,  ahead  of  such  uncertainty  and  risks.  Therefore,  when  these 
risks  become  known  over  the  passage  of  time,  actions,  and  events,  the  analysis  should 
be  revisited  to  incorporate  the  decisions  made  or  to  revise  any  input  assumptions. 
Sometimes,  for  long-horizon  projects,  several  iterations  of  the  real  options  analysis 
should  be  performed,  where  future  iterations  are  updated  with  the  latest  data  and 
assumptions.  Understanding  the  steps  required  to  undertake  an  integrated  risk 
management  analysis  is  important  because  it  provides  insight  not  only  into  the 
methodology  itself  but  also  into  how  it  evolves  from  traditional  analyses,  showing  where 
the  traditional  approach  ends  and  where  the  new  analytics  start. 
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Figure  A.9.  Integrated  Risk  Management  Process 


Conclusion 


Hopefully  it  has  now  become  evident  that  DoD  leadership  can  take  advantage 
of  more  advanced  analytical  procedures  for  making  strategic  investment  decisions 
and  when  managing  portfolios  of  projects.  In  the  past,  due  to  the  lack  of 
technological  maturity,  this  would  have  been  extremely  difficult,  and,  hence, 
businesses  and  the  government  had  to  resort  to  experience  and  managing  by  gut 
feel.  Nowadays,  with  the  assistance  of  technology  and  more  mature  methodologies, 
there  is  every  reason  to  take  the  analysis  a  step  further.  Corporations  such  as  3M, 
Airbus,  AT&T,  Boeing,  BP,  Chevron,  Johnson  &  Johnson,  Motorola,  and  many 
others  have  already  been  successfully  using  these  techniques  for  years,  and  the 
military  can  follow  suit.  The  relevant  software  applications,  books,  case  studies,  and 
public  seminars  have  been  created,  and  case  studies  have  already  been  developed 
for  the  U.S.  Navy.27  The  only  barrier  to  implementation,  simply  put,  is  the  lack  of 
exposure  to  the  potential  benefits  of  the  methods.  Many  in  the  military  have  not  seen 
or  even  heard  of  these  new  concepts.  This  primer,  if  it  is  successful,  serves  to  reveal 
the  potential  benefits  of  these  analytical  techniques  and  tools  that  can  complement 
what  leadership  is  currently  doing.  In  order  to  be  ready  for  the  challenges  of  the  21  st 
century,  and  to  create  a  highly  effective  and  flexible  military  force,  strategic  real 
options,  KVA,  and  risk  analysis  are  available  to  aid  leadership  with  critical  decision¬ 
making.  Real  options  and  KVA  are  tools  that  will  help  ensure  maximum  strategic 
flexibility  and  analysis  of  alternatives  where  risks  must  be  considered. 


27  See  www.realoptionsvaluation.com  (Download  site)  for  more  details  on  the  software  applications 
Risk  Simulator  and  Real  Options  SLS,  as  well  as  sample  case  studies,  videos,  sample  models,  and 
training  seminars  (e.g.,  the  4-day  Certified  Risk  Analyst  public  seminars  cover  all  the  methodologies 
outlined  in  this  primer  and  more). 
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